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Subject Review Handbook 

Al Hussein Fund for Excellence 

Al Hussein Fund for Excellence was established in 1999 under the patronage of King 
Abdullah II. It is a not-for-profit organisation dedicated to promoting excellence and 
innovation among individuals and groups in both the public and private sectors in The 
Hashemite Kingdom of Jordan. The Fund has initiated pilot projects to develop robust 
internal and external assessments of the quality of higher education in the Kingdom. 

Preface 

/ am pleased to be able to introduce this Subject Review Handbook. It not only represents an 
important stage in the development of quality assurance process in Jordan, but also 
demonstrates the success of the Hussein Fund for Excellence in piloting review processes in 
Jordan since 2000. 

The subject review programme is a vital part of the reform process of higher education in the 
Kingdom. The number of voluntary participating universities in the pilot projects has 
testified to the stimulus the subject review method provides for the participants to self- 
evaluate their educational programmes, to seek improvements in quality of various subjects, 
and to open their doors to external commentary by specialist peer reviewers. Furthermore, it 
is also apparent to all involved in the early subject reviews that a review method must be 
clearly documented. 

A Handbook does more than provide a technical description of the method and guidance; it 
also contributes to the credibility of the process by assuring openness and transparency. The 
Handbook represents valuable information drawn from the pilot schemes and the long 
association with the Quality Assurance Agency for Higher Education in the United Kingdom 
(QAA). It also reflects best practice implemented worldwide and, to maintain continuity, I 
envisage that this will be first edition of a series of Handbooks in Jordan that will support 
institutions in their quality assurance efforts. 

Dr. Umayya Toukan, 
Chairman, 

Hussein Fund for Excellence 
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Introduction 

1 . The purpose of this Subject Review Handbook is to describe the method and procedures for 
carrying out subject reviews in Jordan. It also describes for reviewers and institutions the protocols, 
procedures and practical organisation of subject reviews from advanced planning through to the 
publication of reports. The focus of subject reviews is the educational programme(s) in a cognate 
subject area that leads to an academic award, generally a bachelor's degree. 

2. Al Hussein Fund for Excellence (HFE) began pilot reviews in association with the Quality 
Assurance Agency for Higher Education in the United Kingdom (QAA) in 2001 in Computer Science. 
This was followed by two programmes of reviews in Business Administration from 2003 to 2005. 
Following these successful initiatives, the HFE has consulted widely and will continue a schedule of 
reviews with periodic follow-up to check the sustainability of implementation of quality assurance 
processes and standards. Its activities will be conducted in association with all institutions of higher 
education and the national community institutions that represent professional and vocational interests. 

3. Subject review plays a substantial part in meeting the national policy to reform higher education. Its 
strategic aims are: 

• To provide an independent assessment of the quality and academic standards of 
higher education and to encourage a speedy rectification of any major shortcomings 
which may be identified 

• To support the development of systems and processes within institutions of higher 
education that include self-evaluation, internal reporting and continuing improvement 

• To encourage the quest for excellence by awarding prizes for excellence 

• To provide, through the publication of reports, effective and accessible public 
information on the quality of higher education. 

4. The subject review method presented in this Handbook is based on the tried and tested methods 
used in the pilot reviews in Jordan and over many years in the UK and elsewhere in the world, 
However, it has also been developed to reflect the needs and the context in Jordan and the Middle 
East generally. The method and associated protocols in this Handbook encourage institutions to 
address: 

a. The explicit treatment of academic standards. 

b. The use of programme specifications. 

c. An increased emphasis on the use of external reference points to confirm and verify 
the academic standards specified and attained. 

d. A new emphasis on the effectiveness of the institution's arrangements for assuring 
the quality of its educational programmes, including internal and external verification 
of academic standards, engagements with its stakeholders (including students) and 
evidence of the impact of improvement plans. 

e. Modifications to the descriptors for judging the institution's degree of success in 
meeting its stated aims. 

f. Arrangements for a follow-up to each review not less than two years and not more 
than five years after the site visit. 

5. Taken as a whole, these changes move subject review in the direction of a more outcomes-based 
approach, reflecting international trends. 

The Subject Review Method 
Introduction 

6. The subject review method evaluates the quality of educational provision within a subject area as 
defined by a unit of review. Educational programmes of study at all levels and all modes of 
attendance (full-time, part-timer and distance learning) may be reviewed. The HFE will invite the 
institutions participating in the review programmes to put forward for review one or more programmes 
leading to the award of a degree and this will normally include an undergraduate programme. The 
review is focused, at the level of the subject, on the appropriateness of the academic standards set 
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and attained, the quality of learning opportunities, and the adequacy of the institution's arrangements 
to assure the quality and academic standards of the programme(s), measured against the institution's 
mission and the programme specification(s). English is the language of self-evaluation, review and 
reporting. The HFE will inform the institution of the set of key documents that it should make available 
in English for the visiting reviewers. 

The Main Features of the Subject Review Method 

7. The following are the main features of the method: 

• Fitness for Purpose model of quality assurance 

• Programme specification 

• Self-evaluation 

• Review against the subject provider's aims and its programme specifications 
including stated intended learning outcomes 

• Six aspects of provision 

• Peer review 

• Institutional representative (facilitator) 

• Site-visit by external review team 

• Graded profile 

• Overall summative judgement 

• Subject review report 

• Eligibility for an Award for Academic Excellence by the HFE. 

• Follow-up to be arranged jointly by the Fund and the institution 

Fitness of Purpose Model 

8. The review method employs the "fitness-for-purpose" model of quality assurance. This seeks to 
evaluate the appropriateness of the educational programmes and the measures taken by the 
institution to ensure their relevance, currency and effectiveness. It contrasts with alternative models 
that may require compliance with centrally determined sets of aims, teaching programmes and 
examinations. However, where there are reference points and other requirements for determining or 
seeking recognition of the educational programme, such as accreditation requirements by the Ministry 
or by international or professional associations, these are also treated in subject review as part of the 
institution's responsibility to use external reference points effectively to determine and confirm 
relevance and currency. The fitness-for-purpose model places responsibility for quality and academic 
standards on the institution. This responsibility extends to the accountability of the institution for the 
choices it makes on the aspects of provision and the rigour with which it evaluates its own 
performance and seeks continually to improve. 

Programme Specification 

9. The institution is invited to prepare a specification of each educational programme to be reviewed 
(Annex B) setting out the aims of the programme(s), the intended learning outcomes (ILOs), the use 
made of external reference points in confirming the currency, relevance and appropriate level of the 
academic standards of these ILOs, a summary of the strategies adopted for teaching, learning and 
assessment, together with an indication of the range of learning resources specified. A template 
(Annex B1) is provided for institutions embarking on this approach to quality assurance for the first 
time, though compliance with this template is not required providing the institution can demonstrate 
that an equivalent, effective means of specifying the programme and making it known to students, 
academic staff and others, is in place. Annex B also offers a template (Annex B2) for institutions to 
adapt to standardise their specifications for modules or courses within programmes. 

Self-Evaluation 

10. Self-evaluation is central to quality assurance and to the subject review method. The institution is 
invited to undertake this as a start to the subject review process. There should be one self-evaluation 
report (SER) for each subject review, even if there is more than one educational programme being 
reviewed. The SER should be an analytical, evaluative, evidence-based account providing the context 
in which the programme(s) is/are provided, the distinctive features and an assessment of the 
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strengths, weaknesses, opportunities and threats. It should include, as an attachment, a programme 
specification for each programme. Although it will be strategic in its evaluation, it is also expected to 
focus on the interests of recent, current and prospective students. The SER should be organised 
within the structure given by the six aspects of provision. The SER will be presented to the peer 
reviewers and forms the basis of the site visit. It should discuss in a balanced way both strengths and 
weaknesses in the provision. Where weaknesses are acknowledged, the subject provider is 
encouraged to discuss the issues and the steps taken to address them. Guidance on the conduct of 
self-evaluation and the preparation of the SER, including the presentation of statistical data on 
student progression and achievement, is set out in (Annex C). 

Review using the subject provider's stated aims, including intended 
learning outcomes 

1 1 . The internal self-evaluation and the external review are carried out using the stated mission of the 
institution and educational aims of the educational programme(s) to be reviewed. This "mission- 
sensitive" approach places responsibility upon the institution to articulate clearly its aims and to 
demonstrate the extent to which it is successful in meeting them. The ILOs contained in each 
programme specification require care not only in their development, but also in their implementation, 
so that they are appropriately reflected in the arrangements made to provide the learning 
opportunities and assess the level of achievements of the students. Reviewers evaluate the extent to 
which each of the six aspects of provision contribute to meeting the programme specification(s) and 
thus support the students' opportunities for learning and their achievement. Reviewers also evaluate 
whether the academic standards set and attained allow the aims set by the subject provider to be met 
and support the institution's mission. 



The aspects of provision 

12. The aspects of provision provide a common structure for each stage of the subject review 
process: the self-evaluation report, the site-visit, the judgements and the subject review report. The 
six aspects of provision are: 

Curriculum Design, Content and Organisation (CDCO) 
Teaching, Learning and Assessment (TLA) 
Student Progression and Achievement (SPA) 
Student Support and Guidance (SSG) 
Learning Resources (LR) 

The Effectiveness of Quality Management and Enhancement (EQME). 

13. It is intended that the aspects of provision should be sufficiently broad and inclusive to enable 
institutions and reviewers to give full weight to the particular features of the provision in different 
contexts. Broad indications of what might be addressed within each aspect are listed in the Subject 
Review Aide Memoire (Annex A). This aide-memoire differentiates between the key features of each 
aspect and matters which interrelate and influence more than one aspect of provision. Institutions 
may address matters in addition to those listed in the Subject Review Aide Memoire in order to 
provide an evaluation that is appropriate to the specific provision and context. 

Peer Review 

14. Reviews are carried out by a team of subject-specialist reviewers, led by a review chair. In the first 
place, the peer reviewers are introduced to the HFE by the QAA in the UK because of their extensive 
experience in subject review and evaluative processes. They are appointed by the HFE in line with 
published criteria and operate independently using the published method and protocols (Annex D). It 
is the subject-specialist reviewers' main responsibility to gather evidence and to make judgements on 
the quality of education provided. Subject specialist reviewers are drawn mainly from the higher 
education sector, although nominations and applications from industry, consultancy and the 
professions are also considered where their experience and insights are helpful and where they also 
have direct knowledge of higher education. Subject-specialist reviewers and the appointed review 
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chairs are briefed on the review method and the context of higher education in Jordan by the HFE in 
collaboration with the QAA before their first review visit. 



The Institutional Representative (facilitator) 

15. The HFE will invite the institution to nominate a senior member of the institution to take the role of 
the institutional facilitator for each review. The role is described in (Annex N). The role begins with the 
early phases of preparation for self-evaluation and will normally end with the processing of the subject 
review report. However, at the discretion of the institution, the facilitator may play a significant 
facilitating role in any subsequent developments arising from the recommendations and any actions 
that are planned to follow the subject review report. 

The site-visit 

16. The main purposes of a review site-visit are to enable reviewers, having read the SER, to gather 
further evidence, on the educational programme(s), sufficient to allow them: 

• To form a collective judgement on the appropriateness of the academic standards set 
and attained, the quality of learning opportunities, and the adequacy of the 
institution's arrangements to assure the quality and academic standards of the 
programme(s), measured against the institution's mission and the programme 
specification(s) 

• To establish a graded profile and an overall judgement on the quality of that provision 

• To prepare a draft subject review report. 

17. Advanced planning of the site-visit is critical to a successful review. The typical visit schedule 
(Annex F) offers equitable treatment to all institutions and a degree of flexibility in arranging activities 
to suit particular circumstances. Some activities are essential to an effective external review and 
others are at the discretion of the review chair. The review chair will agree the programme of activities 
with the institution at least two weeks before the site visit is due to commence. 

The Graded Profile 

18. The graded profile shows the extent to which the institution has established and implemented its 
aims and ILOs, and provides an appropriate programme leading to an academic award that is quality 
assured, meets the stated aims and supports the mission of the institution. The profile is informed by 
the programme specification(s), the SER and the evidence base generated for and by the site-visit. 
The profile is created by applying a grade from 1 to 4 in ascending order of quality to each of the six 
aspects of provision. The grade descriptors are set out in (Annex L), and their assignment is a matter 
for the professional judgement of the team. Each subject review leads to a single graded profile even 
if more than one educational programme is included in the review. However, the reviewers may wish 
to differentiate between the performance of programmes within their evaluation and in the conclusions 
presented in the subject review report. 

19. A grade of 2 or better for an aspect means that it makes at least an acceptable contribution to the 
attainment of the stated ILOs and that the aims are at least broadly met. A grade of 1 means either 
that the aspect does not make an acceptable contribution to the attainment of the stated ILOs, or that 
the level of attainment of the ILOs does not allow the aims set by the institution to be met. A grade of 
3 or higher for CDCO, SPA and EQME indicates that the institution has demonstrated that this aspect 
is verifiable against comparable programmes in the Kingdom, regionally and worldwide for the 
academic standards represented in the degrees awarded by the institution. A grade 4 in any of these 
three aspects demonstrates a higher order of performance that (while still able to benefit from 
continuing improvement) is at least the equal of any similar programme, verified by the use of external 
referencing and evaluation. 
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The Overall Summative Judgement 

20. The overall summative judgement is derived from the profile. Each aspect has equal weight. A 
profile with all aspects graded 2 or better will be reported as 'quality approved'. A profile with two or 
more grade 2s will result in an invitation from the HFE to the institution to provide an improvement 
plan and to demonstrate in due course its impact. A profile with one or more aspects of provision 
graded 1 will be 'subject to further review within a year'. The summative profile that contains at least 
four grade 4 and no grades lower than 3 will be eligible for consideration by the HFE for its Award of 
Academic Excellence (please see below). The judgements made by the peer review team are tested 
by the advisor to the HFE on behalf of the QAA and the consultant appointed to the subject review 
programme, in session with the review chairs. The test for these outcomes comprises a check: that 
the review process has conformed to the published method and protocols; and that the judgements 
are evidence-based, in line with the evaluations presented in the main text of the subject review 
report, and consistent with the descriptors contained in this Handbook. An outline of the main 
conclusions is provided to the institution by the review chair on behalf of the peer review team at the 
end of the site-visit. However, the graded profile is not given to the institution until the outcomes are 
tested and verified after the site-visit. 

The Subject Review Report 

21 . A subject review report is prepared after each review and is the main documented outcome of the 
review process. The institution will receive a draft of this report shortly after the review is completed, 
and will be invited to confirm its factual accuracy. The institution's comments, together with the test 
conducted by the HFE, will assist the HFE in formally adopting the report as a fair, balanced 
evaluation with conclusions supported by evidence. The HFE will wish to publish on its website a 
summary report for each review, setting out the arrangements for the review with a summary of the 
outcomes. This will be accompanied on the website by an overview report, drawing conclusions from 
the set of subject reviews in the subject. 

Eligibility for the Award of Academic Excellence 

22. The HFE, in line with its mission to promote excellence, wishes to identify institutions which 
demonstrate the highest levels of performance to the benefit of their students and wider society. It will 
invite the review chairs, together with the consultant appointed to the subject review programme and 
the advisor to the HFE on behalf of the QAA, to present the outcomes of the review programme with a 
recommendation for the Board of the HFE to consider making an award for exceptional performance. 
This recommendation will be based on the verified subject review reports and will require a graded 
profile not less than four grade 4s and no grades lower than 3. The decision of the HFE will be final 
and will be announced shortly after the end of the schedule of subject reviews. 

Follow-up review 

23. The HFE will arrange with the institution to follow up each subject review at an appropriate time, 
not less than two years and not more than five years following the date of the site visit. The prime 
purpose of this follow-up is to evaluate the progress made since the subject review by measuring the 
impact of the review process on the quality of the subject and the academic standards set for, and 
achieved by, the graduates. The follow-up process is designed to support institutions in their 
development of internal systems and processes for sustainable improvement. The follow-up review 
will lead to a report which will contribute to the HFE's overall evaluation of the impact of subject 
reviews. Both the individual follow-up review report and the overall evaluation of the impact of subject 
reviews will be published on the HFE's website. 
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The Subject Review Process 
Preparation for Subject Review 
Introduction 

24. The preparation for review may be divided broadly into five stages: 

1 . Advanced planning by the HFE and the institution to establish a timetable and arrange for briefings. 

2. The self-evaluation by the institution. 

3. The composition of the review team. 

4. The analysis of the self-evaluation report together with the preparations by the review team 
including reading the self-evaluation report and writing initial commentaries. 

5. Preparatory discussions between the review chair and the institution concerning practical 
arrangements for the site visit and documentation. 

25. The preparations for review are most effective when they are undertaken with full co-operation 
between the institution (including the facilitator), the officers of the HFE, the review chair and the 
subject specialist reviewers. The subject review consultant appointed by the HFE to lead and support 
each schedule of subject reviews plays a key role in the advanced planning, the preparations and the 
implementation of the schedule. 

Advanced planning 

26. Advanced planning is undertaken by the HFE in conjunction with the institutions. As part of the 
early planning of a review programme, the HFE will announce its proposed schedule, appoint a 
consultant to the review programme and arrange with institutions a phase of discussions and 
consultations. These will concern the scope of the reviews, the context within which the method is to 
be applied and any special circumstances to be taken into account. The precise timing of review 
visits is a matter for discussion and agreement between the institutions and the HFE. 

27. The HFE will invite the institution to present for review one or more educational programmes 
leading to a degree. Normally, where the subject as defined is homogeneous, only one programme 
will be selected and this will be its major undergraduate programme in the subject. Where the subject 
contains a number of discreet disciplines, the HFE, taking the advice of the subject review consultant, 
will consult with institutions on the most appropriate means of capturing the range of disciplines within 
a sample of the programmes. The HFE may also ask the institution to supply information on the size, 
scope and nature of the provision to be reviewed, the institution's preferred dates for the review visit 
and any dates that are unsuitable. This advance information provides a basis for further discussions 
with institutions to plan and agree the timing of individual visits, and informs the composition of the 
review team. Under the terms of the Memorandum of Understanding between the HFE and the QAA 
in the UK, the HFE will draw upon trained and experienced subject specialists as peer reviewers. The 
HFE will also, if necessary, appoint an interpreter/translator who will assist the institution and the 
visiting review team by providing translation of key discussions and impromptu translation of key 
supplementary papers that had not been made available in English beforehand. The HFE will arrange 
appropriate briefing for institutions, reviewers and interpreters/translators. 

Self-evaluation 

28. Institutions are asked to undertake their self-evaluation in the period starting at least six months 
before the date of the planned site-visit. Guidance on the process of self-evaluation and the SER is 
provided in (Annex C). The HFE will give the date for the submission of the SER to the institution. As 
a guide, this will normally be approximately two months before the schedule of site-visits. 
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The review team 

29. The visiting review team will normally consist of not less than two subject specialist reviewers and 
not more than three, and a review chair who will normally be, but need not be, a specialist in the 
subject. This will vary exceptionally to accommodate any unusually large and complex degree 
programmes. The team will be composed by the HFE with advice from the advisor on behalf of the 
QAA and from the subject review consultant appointed by the HFE. The HFE will notify the institution 
of the team not later than one month before the site-visit. Although the institution has no veto or 
choice in the selection of the review team, the HFE welcomes any additional comment from the 
institution on the suitability of the review team such as any potential conflict of interest so as to ensure 
that the profiles of the team and the institutional staff are a reasonable match. 

30. A review chair leads each review team and is responsible for co-ordinating and managing the 
review visit, and for ensuring that it is conducted within the HFE's guidelines. This involves: preparing 
for the review visit in liaison with the institution; co-ordinating the work of the subject specialist 
reviewers on the review team; ensuring that evidence is gathered and tested and that the judgements 
reached are robust; providing oral feedback to the institution at the end of the visit; and preparing the 
subject review report after the visit. 

31. The responsibilities of subject specialist reviewers include: analysing the self-evaluation report 
and preparing an initial commentary; participating in the review visit to gather, share and test 
evidence and to make judgements on the quality of education, contributing written draft paragraphs 
for the subject review report; commenting to the review chair on the draft subject review report after 
the visit. 

Analysis of the Self-Evaluation Report (SER) 

32. The SER provides the basis for visiting reviewers to set the priorities and the programme for the 
review visit. The HFE will forward the report to the review team after checking that the report 
addresses the agreed educational programme(s), follows the structure for the report as set out in this 
Handbook and contains the programme specification(s). The review chair analyses the SER, using 
the Subject Review Aide Memoire (Annex A), before contacting the institution and the other review 
team members. In order to achieve the best possible preparation for the review visit, the review chair 
may wish to clarify with the institution their understanding of the self-evaluation, with particular 
references to the external reference points for academic standards, the clarity and completeness of 
the programme specification(s), the supporting statistical information and the range of teaching, 
learning and supporting learning resources addressed within the SER. 

33. The review chair also contacts the subject specialist reviewers to discuss arrangements for the 
visit and to agree the responsibilities within the team. Subject specialist reviewers normally assume 
responsibility for co-ordinating and sharing evidence relating to two or more aspects of provision. The 
review chair will arrange for the subject specialist reviewers in the review team to prepare initial 
commentaries on the self-evaluation and these initial commentaries will be e-mailed to team members 
and the facilitator at least one week before the site-visit commences. 

34. In producing these commentaries, reviewers should refer to the questions in the Subject Review 
Aide Memoire (Annex A). The commentaries should make full reference to academic standards, the 
programme specification(s) and identify matters on which further evidence is required. The initial 
commentaries form the basis of the discussion in the first team meeting of the review team to which 
the facilitator is invited. From this discussion will emerge confirmation of the appropriate focus for the 
reviewers' activities during the site-visit and the provisional lines of enquiries to follow at meetings to 
be held in the site-visit programme. 

Practical arrangements for the site-visit 

35. The HFE will arrange with the institution to make known to the review team the names, job titles 
and email addresses of the senior academic member(s) of staff who is/are a central institution 
contact(s), of the person(s) responsible for the educational programme(s) being reviewed and the 
SER, and of the facilitator. The review chair will use e-mail to introduce the reviewers and his/herself 
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to the institution. This exchange should normally take place not less than three weeks before the 
review visit. Matters that might be raised at this stage will include: 

• Any necessary clarification of the SER and any feedback on the completeness of the 
self-evaluation that might assist the institution to make further preparations for the 
visit 

• The role of the institutional facilitator 

• the range of students' assessed work available for scrutiny and the extent to which a 
representative sample can illustrate the range of student achievement in line with the 
intended learning outcomes in the programme specification(s) 

• The range of teaching and learning sessions available for observation during the visit, 
and the extent to which such observation may be required and feasible 

• The documentation to be sent to reviewers prior to the review visit, the timetable for 
its receipt, and the documentation to be made available during the visit 

• meetings and other activities to be arranged in advance 

• Other practical arrangements for the visit such as the room and facilities for the 
review team. 

36. The review chair will then write to the institution (with a copy to the HFE), in order to confirm the 
practical arrangements agreed for the review visit. 

37. The review process requires the institution to make available accurate, representative and 
accessible information and evidence. Annex E summarises the range of documentation and 
information which is usually provided prior to and during a review visit, and provides guidance on the 
sample of student work to be provided. The overall volume of material provided in the base room 
should be manageable within the scope of a two-and-a-half-day visit. The student work provided 
should be sufficient to secure a reasonably representative sample, as indicated in Annex E. In most 
cases, institutions will be able to identify appropriate samples from the materials that are kept 
routinely for examination purposes. However, where relevant, samples of recently marked work that 
do not contribute to final assessment may be requested. 

38. The HFE will advise the institution on the key documents that should be provided in English for 
the visiting review team, and will make arrangements with the institution and the interpreter/translator 
it appoints to the review site-visit for the translation of any other documents in the period running up to 
the site-visit. 

39. Responsibility for making travel arrangements to Jordan rests with the reviewers. The HFE will 
confirm with the review team members the arrangements for transfer from the airport to the hotel and 
the accommodation. Daily transport between the hotel and the institution is generally arranged by the 
institution at times agreed with the review chair. 

40. The base room provided in the institution should accommodate the visiting team together with the 
facilitator and translator. Either within, or adjacent to, the base room the institution will provide 
supporting documentation including the samples of students' assessed work. Light refreshments with 
bottled water should also be provided. The institution will also wish to set aside suitable rooms for the 
programme of meetings to be agreed as part of the site-visit schedule. Other facilities to be discussed 
in detail will include access to telephones and photocopying equipment. Subject specialist reviewers 
who bring laptop computers will find them helpful to compile and transfer written summaries 
electronically. However, the institution may also wish to provide a web terminal in the base room to 
support the review, particularly when the institution has a website and intranet facility that provides 
access to detailed relevant information. 
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The Subject Review Visit 

The Review Process 

The Purposes of a Review Visit 

41. The review visit is conducted in a spirit of dialogue and co-operation between the institution, the 
subject staff, institutional facilitator and the review team. The main purposes of a review visit are: 

1 . to enable reviewers to gather sufficient evidence, starting with the SER, on the educational 
programme(s) to allow them to form a collective judgement on the appropriateness of the academic 
standards set and attained, the quality of learning opportunities, and the adequacy of the institution's 
arrangements to assure the quality and academic standards of the programme(s), measured against 
the institution's mission and the programme specification(s); 

2. to establish a graded profile and an overall judgement on the quality of that provision; and 

3. to prepare a draft subject review report. 

42. A review visit normally lasts two-and-a-half days. A common range of activities is undertaken 
during visits, although the precise order and balance may be adapted to reflect the nature of the 
provision. 



The Programme for a Review Visit 

43. An outline programme is agreed between the review chair and the institution as part of the 
preparations for the site-visit. A typical schedule for a programme is set out in Annex F. The activities 
carried out by a review team during the visit include the following: 

• scrutiny of institutional and programme documents, internal reviews and reports 

• sampling of student work (examination scripts, coursework, projects, artefacts and 
dissertations) 

• at the discretion of the review team, a small sample of class observations reflecting 
the range of teaching and learning being carried out during the review visit (this may 
include direct observation of classes, seminars, workshops, tutorials, practicals and 
placements as appropriate) 

• meetings with academic and administrative/support staff 

• meetings with students, former students and, where appropriate, employers 

• an overview of the learning resources available to students on the programme(s) 

• meetings of the review team to consider the evidence, share information and form 
judgements. 

44. The review chair maintains an overview of the range and balance of review activities, and guides 
subject specialist reviewers in apportioning their time. The achievement of an appropriate balance 
between sampling student work, observing teaching and learning, reading documentation, meeting 
staff, students and other stakeholders, producing written summaries and arriving at a collective 
judgement within an intensive visit, requires careful planning and co-ordination. It is essential that the 
overall range and the balance of activities undertaken enable the team to develop a robust evidence 
base in order to make sound judgements made in relation to each of the six aspects of provision. 
Subject specialist reviewers are expected to prepare and agree individual timetables with a view to 
achieving this evidence base. In line with the emphasis on academic standards and learning 
outcomes, reviewers should allocate sufficient time for reading students' assessed work. 

45. Where possible, the review team with the facilitator should meet before attending the institution, 
usually in the hotel on "day zero", that is the day before the site-visit is due to start. When two site 
visits are scheduled to run end-on with the same review team, the ideal arrangement is to arrange 
both "day zero" meetings for the two site-visits on the day before the first site-visit. At this meeting, it 
can be helpful if the institutional facilitator informs reviewers of any matters, from the institutional 
perspective, which may be important to their understanding of the programme being reviewed. This 
session may also be used to meet the head of the institution, the head of the faculty or school in 
which the programme is offered and/or other senior representatives. The HFE may, depending on the 
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schedule of site-visits, wish to organise these sessions into a single formal reception of 
representatives of the institutions and the visiting reviewers. 

46. An initial meeting with available subject staff at the beginning of the first day of the visit allows an 
opportunity for the staff representatives to make a brief, concise, presentation (no more than 10 
minutes) on the programme(s) being reviewed and the context, and to inform reviewers of any 
developments since the self-evaluation was undertaken. This first meeting with staff also provides an 
opportunity for the review chair to remind staff, with team members present, of the subject review 
method and its protocols. The institution may invite student representatives to participate in this initial 
meeting. 

47. The review visit schedule includes a range of meetings between members of the institution and 
the reviewers to consider the various aspects of the provision. Throughout the visit, the review team 
meets daily to discuss its visit schedule and its findings. The institutional facilitator is an observer at 
most meetings and may provide factual information relevant to the team's discussions. However, the 
institutional facilitator may not attend the meeting with students or parts of meetings which involve 
direct discussion of grades, such as the final reviewers' team meeting. 



Gathering Evidence 
General Considerations 

48. In gathering evidence, reviewers should use the SER as a base line, supplemented by supporting 
evidence provided by the institution and further evidence collected during the site-visit. The reviewers 
and the institution should refer to the Subject Review Aide Memoire (Annex A) to ensure that the 
range of enquiries addresses the full spectrum of the quality of learning opportunities and the 
academic standards of the programme. All reviewers are expected to identify, share, consider and 
evaluate evidence related to all aspects of provision. 

49. Reviewers make notes of meetings with staff and students, of teaching and learning observations 
and of comments on the assessment and quality of student work. The collation and circulation of 
notes within the review team assist in developing a collective evidence base for the judgements 
made. A suggested template for making meeting notes is given in (Annex J). Reviewers refer to 
these notes and their initial commentaries on the self-evaluation in preparing written evaluations of 
the quality of provision; the notes are retained by the review chair at the end of the visit. 

Sampling Student Work 

50. The sampling of student work is an important dimension of the activities undertaken during a 
review visit. This activity directly supports the evaluations made in respect of the academic standards 
achieved as reflected in Curriculum Design, Content and Organisation (CDCO) Teaching Learning 
and Assessment (TLA) and Student Progression and Achievement (SPA). It also contributes to the 
evaluations made under Student Support and Guidance (SSG), and the Effectiveness of Quality 
Management and Enhancement (EQME). Subject review requires reviewers to evaluate: 

• the relevance and currency of the ILOs, as verified by the institution in its use of 
external reference points to confirm its academic standards in comparison with 
equivalent programmes elsewhere (sometimes referred to as benchmarking) (CDCO) 

• the appropriateness of the arrangements for the assessment of students' attainment 
in line with the programme specification(s) and in particular the ILOs (TLA) 

• students' achievement of the learning outcomes (SPA) 

• the contribution that assessment makes to students' learning (sometimes referred to 
as formative assessment) (TLA) 

• the appropriateness of the marks awarded (SPA) 

• the extent to which marking is internally and externally verified (SPA and EQME) 

51 . Details of the sample of student work required for the purposes of review are provided in Annex E. 
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52. Judgements about the students' achievements are mainly based on the documented evidence of 
external reference points, the sampling of student work, the series of indicators or statistical 
information and observations of teaching and learning. The reviewers record their comments on the 
assessment design, standards achieved, marking and feedback to students in the Student Work and 
Assessment Note (Annex I). Where the arrangements made by the institution include oral feedback to 
students instead of, or in addition to, a written commentary, and information on this process is 
presented to them, the reviewers will take this into consideration. 



Observation of Teaching and Learning 

53. Observation of classes can provide valuable insights into learning and teaching, and may take 
place at the discretion of the review chair. However, any observations should be kept to a minimum in 
an intensive site-visit. In addition, the reviewers and the institution will wish to consider the extent to 
which, where the language of instruction may not be English, the evidence gained can be viewed as 
sound and representative. It is therefore recommended that class observations, if undertaken, should 
represent no more than the equivalent of two hours per subject specialist reviewers and agreed in 
advance with the professor or teacher as being feasible. The aim of direct class observation is to add 
to the evidence provided in the self-evaluation on the quality of the student learning experience in 
order to improve the reviewers' understanding of the range of learning opportunities available. A good 
self-evaluation report is likely to present a summary of information from internal peer reviews of the 
quality of a sample of classes and this may be considered by the visiting review team to be an 
adequate evidence base without the need for further observations during the site visit. Class 
observations in peer review are not a form of appraisal of the performance of the academic member 
of staff. If class observations are to take place in a site-visit, the review chair will prepare a preliminary 
schedule of observations for discussion with the review team and then discuss this with the named 
contact for the review and/or the head of the faculty or school. 

54. The reviewer should meet the member of staff responsible for a teaching and learning session 
before it commences in order to introduce himself or herself, to discuss the overall intended learning 
outcomes related to the activity, to identify the particular intended contribution of this session and to 
determine how students are expected to learn from it. Understanding the overall purpose of the 
teaching and learning session is highly important. For example, a lecture delivered for the express 
purpose of transmitting information will be structured differently from a session designed to develop 
specific skills or encourage the application of skills by the students. Reviewers will not make 
comments during a lecture, seminar or tutorial and should not be intrusive or engage directly in the 
teaching or learning. In the case of long sessions (more than an hour), an appropriate period or 
periods of observation should be agreed, if possible, at the preparatory meeting. On occasion, 
students engaged in learning activities in practical sessions and independent learning may be asked 
by reviewers to talk about their learning experiences and how the activity being observed fits within 
their wider programme of study. 

55. A standard Observation Note (Annex H) is completed for each teaching and learning session 
observed. In making their judgements about an individual session, reviewers evaluate whether the 
teaching and learning approaches and the materials used are effective in achieving the intended 
learning outcomes. This includes consideration of any relevant written or computer-based guidance 
for students and samples of student work where these are available. This Observation Note may also 
be used for internal peer review at the discretion of the institution. 

56. After attending a session, the peer reviewer offers a brief oral feedback to the member of staff, 
even if this includes a later appointment being made to provide the feedback. This oral feedback is 
confidential to the member of staff, and is always given privately. The purpose of the feedback is to 
offer constructive comment on the observations made rather than to prescribe preferred practice. 
Reviewers must preserve the anonymity of staff in all written reports and in discussions with other 
members of the institution. 
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Learning Resources 

57. Reviewers also gather evidence through direct examination of the learning resources accessible 
to the students on the programme(s) under review. Reviewers normally visit the facilities made 
available to the subject, and may observe students and staff using accommodation, specialist IT or 
other equipment in the course of normal teaching and learning activities. In looking at library 
provision, reviewers may undertake catalogue searches or request access to on-line facilities. In 
evaluating the quality of learning resources, reviewers' direct observations of facilities are considered 
alongside evidence from student work, written documentation, meetings with relevant staff, and 
meetings with students. The emphasis is on access and use of facilities by students in the subject. 
The review of learning resources may inform judgements in relation to other aspects (such as 
Curriculum Design, Content and Organisation, and Teaching, Learning and Assessment), which are 
directly affected by the quality of available resources. 

Meetings with Staff 

58. As a general principle, reviewers focus their attention at the level of the subject. The review team 
addresses matters at a central institutional level only when necessary because of their impact on the 
learning opportunities for students in the subject. It may be helpful to structure meetings with staff 
around aspects of provision, perhaps in combination. For example, the typical outline site-visit 
schedule (Annex F) suggests that academic standards are addressed taking more than one aspect in 
one meeting. In another instance, reviewers might meet jointly with academic counsellors, personal 
tutors and representatives of the student welfare services to gain an overview of student support and 
guidance. Alternatively, they might meet jointly with a departmental library representative, subject 
librarian and representative of the central computer services and/or other relevant staff to discuss 
learning resources in the subject. In developing specific agendas for these meetings, reviewers refer 
to the Subject Review Aide Memoire (Annex A) and keep a written record of proceedings. 

59. Daily meetings between the review chair, the institutional facilitator and the departmental 
representative provide an opportunity to highlight particular issues which the reviewers wish to 
explore, and for the institutional facilitator and the departmental representative to raise any matters 
concerning the conduct of the review. Individual reviewers are encouraged to make any request for 
additional documentation or further meetings through the review chair. The review chair will liaise with 
the institutional facilitator and the departmental representative in making these arrangements. 

60. An additional meeting between the review team and subject staff during the latter part of the visit, 
normally in the afternoon of day two or the early part of the final day, provides reviewers with an 
opportunity to identify as far as possible all key matters likely to affect the graded profile and to seek 
clarification of any points still outstanding. It also enables staff to engage in a dialogue about matters 
arising during the review visit. This meeting is not for the interim feedback of conclusions, nor is it the 
final opportunity for dialogue between the review team and staff. 

Meetings with Students, Former Students and Employers 

61. The views of students and others with a legitimate interest in the quality and standards of the 
programme(s) (referred to sometimes as the stakeholders) are important evidence. Whenever 
possible, all of the review team should attend the meeting with students, which typically lasts an hour 
(Annex G). The review chair generally chairs this meeting. At the beginning of the meeting, the review 
chair introduces the review team and provides a brief outline of the subject review method. Those 
attending should be assured that the contributions made will not be attributed to individuals. 
Reviewers should also state clearly that their final evaluation of the quality of provision will be based 
on a wide range of evidence, of which comments made by students form one part. One team member 
or the review chair should take notes and ensure that a summary of the main points is circulated 
among the review team. This summary should not attribute comments to individuals. The institutional 
facilitator does not attend these meetings, but may assist the team in identifying further information in 
order to clarify any issues students raise. 
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62. Discussion should be structured and all present should be encouraged to contribute. If a meeting 
has been arranged over a light buffet, informal discussion should be kept relatively brief and the 
meeting should then be convened formally to ensure proper coverage of key areas. Reviewers are 
seeking the views of students on the six aspects of provision and should ensure that a clear agenda is 
outlined. An aide memoire for meeting with student groups is provided in Annex G; this can be 
adapted to take account of the particular group. Reviewers will be keen to gauge: students' 
awareness of the subject provider's aims and ILOs; their experiences as learners in their programme; 
their responses to the teaching and assessment provided; and their views on academic and personal 
tutorial support, learning resources, and the arrangements made for student feedback and 
representation (including whether or not they were involved in the self-evaluation process). Reviewers 
should allow time for students to raise points. 

63. The review team may also take into account the views of recent former students and employers or 
other representatives from the relevant industry or profession who are able to give an informed 
overview of the provision and its relevance, and of the attributes of the graduates. These important 
stakeholders may also offer their views on the extent to which they are consulted by the institution as 
part of the maintenance and continuing improvement of the programme(s). 

Scrutiny of Documentation 

64. Reviewers also gather evidence through scrutiny of documentation. The evidence base drawn on 
includes evidence of the uses made of external reference points, records of quality assurance 
procedures, reports and advice from external sources, engagements with stakeholders, including 
employers and professional and accrediting bodies, as well as internal documents and reports. 
Emerging judgements are refined and tested against as wide a range of evidence as possible; for 
example, views expressed in meetings by staff or students will be checked and tested against the 
documentation provided. 



Making the Judgements 
General Considerations 

65. The reviewers' collective judgements on the academic standards and the quality of learning 
opportunities in relation to each aspect of provision are informed by all the evidence gathered and 
considered by the whole review team. In each case, judgements are based on the principle of fitness 
for purpose in relation to the stated mission of the institution, the programme specification(s) and the 
extent to which the aims and ILOs are being met. Grades are assigned by the whole review team, on 
the basis of the evidence gathered in relation to each aspect of provision and using the criteria for 
assigning grades (Annex L). All team members are expected to share information gathered which is 
relevant to any aspect of provision, but individual reviewers may co-ordinate the evidence related to 
particular aspects of provision. Written summaries of the information and evidence gathered should 
be distributed continuously among the team, including notes of meetings (Annex J), and completed 
observation and student work/assessment notes (Annexes H and I). 

Review Team Meetings 

66. The review team meets daily to discuss its findings, normally at the end of the afternoon or in the 
early evening. Team meetings are used to review the evidence gathered in relation to each aspect of 
provision, to form or refine preliminary judgements and to determine which issues require further 
exploration. Reviewers are expected to evaluate how the evidence gathered compares with the 
evidence and conclusions contained in the self-evaluation report and to test emerging judgements 
against (a) the strength of evidence gathered and (b) the descriptors used to determine the graded 
outcomes. The review team meeting at the end of the second day should allow sufficient time for a 
discussion of the evidence gathered in relation to each aspect of provision and a preliminary 
discussion of grades. Discussion of the information gathered and emerging judgements should 
involve the whole team. 
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67. The reviewers hold a final meeting on the last day to review any additional evidence, to agree the 
outline of strengths and weaknesses in each aspect of provision, to finalise the grading for each 
aspect of provision and to determine the overall conclusions to be reported to the institution. The 
institutional facilitator does not attend this meeting. 

Written Summaries 

68. The draft summaries written by reviewers during the visit focus on the evaluation of the evidence 
gathered in relation to each aspect of provision. Summaries should be analytical rather than 
descriptive and make direct reference to relevant parts of the programme specification(s), with 
particular attention paid to the aims of the programme(s) and the ILOs. They should also cite the 
sources of information such as the SER (section or page reference), meetings, documents, reviews of 
the sample of students' assessed work and any observations of teaching and learning. The written 
evaluation should summarise the relevant strengths and issues relating to each aspect of provision 
and, overall, should justify the grade assigned. 

Grading the Aspects of Provision 

69. The criteria for assigning grades are included in Annex L. 

Reviewers are required to make judgements on, and agree a grade for, each of the six aspects of 
provision. In assigning grades, reviewers judge the extent to which: 

• The students' learning opportunities and the academic standards set and attained 
support the programme specification(s) 

• Each aspect of provision contributes to the attainment of the aims and ILOs, and 
whether the ILOs set and the level of attainment of those ILOs allows the stated aims 
to be met. 



The Oral Feedback Meeting 

70. The meeting with senior representatives of the institution at the end of the review visit is intended 
to provide oral feedback on the outcomes of the review. The meeting is chaired by the review chair, 
who will convey the key conclusions with strengths and weaknesses and the main evidence that will 
be included in the published report. The grades will not be presented, since these are subject to 
verification by the HFE and the QAA. It is not a consultative meeting which might affect the 
judgements of the reviewers, although a limited amount of clarification by the reviewers and the 
institutional facilitator may take place. The agenda for the oral feedback meeting is set out in Annex 
M. 



The Review Reports 
Subject Review Reports 

71 . A written subject review report is prepared by the review team after each review visit. The report 
includes: a brief description of the review method; a statement prepared by the subject provider of the 
aims for the programme(s) being reviewed; the graded profile and the overall judgement on the 
quality of provision; an evaluation of the quality of education, organised under the six aspects of 
provision; and the conclusions reached. The total length is normally about 4,500 words. The HFE has 
procedures in place for monitoring closely the progress of individual reports. 

72. The subject review report is the main documented outcome of the subject review process and will 
be studied carefully by the institution as well as by others. It provides a permanent record of the 
review. The reports should be primarily evaluative and analytical. Succinct, accurate writing and a 
consistent style are essential. They should focus on the evaluation leading to the judgements made 
using the specified criteria and the evidence which supported them, in relation to the institution's 
mission and the programme specification(s). They should include appropriate statements and 
evaluations for each aspect. The conclusions reached should be evidence-based and logical and the 
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reader should not be surprised by the grades after reading the text related to each aspect. The 
balance and tone of the writing should also reflect the overall conclusions and give due credit for 
strengths as well as weaknesses. 

73. The review chair produces the first complete draft of the report in the week immediately after the 
visit by drawing on the summaries prepared by the subject specialist reviewers during the visit. This 
draft is sent to the subject specialist reviewers, who then check to ensure that it is factually accurate 
and that it represents the views of the review team. It is particularly important that reviewers return 
comments on the first draft of a report to the review chair within one week, in order that s/he may 
prepare a revised draft subject review report within the production schedule for reports. 

74. The subject review reports are scrutinised by the consultant in the review programme and by the 
advisor to HFE on behalf of the QAA. The draft reports are tested against a set of criteria and 
considered by a panel comprising the review chairs, the consultant and the advisor (attended by the 
General Manager of the HFE), before they are verified by the consultant and the advisor on behalf of 
the QAA. These criteria are: that the review process conformed to the published method and 
protocols; that the judgements are evidence-based; that the conclusions are in line with the 
evaluations presented in the main text of the subject review report; and that the conclusions are 
consistent with the descriptors contained in this Handbook. Competent review reports prepared by 
peer reviewers are not edited or altered by the HFE. 

75. The subject review reports are sent to the institution when they have satisfied the internal tests. 
At this stage the institution is invited to comment on any factual inaccuracies in the draft report. The 
review chair finalises the subject review report in the light of comments from the institution and sends 
it to the HFE, copied to the subject review consultant and the advisor to the HFE on behalf of the 
QAA. At present the HFE does not have plans to publish the full subject review reports. 

Summary Review Reports 

76. The review team will also complete a brief summary review report for each review. This will record 
the date of the subject review together with a brief contextual statement of the review method and 
present the overall conclusions and grades for the reviewed programme. The HFE may wish to 
publish the summary review report on its website. 

Subject Overview Reports 

77. The HFE will commission the writing of a subject overview report based on the individual subject 
review reports produced at the end of each schedule of subject reviews. These overview reports will 
be published. They are designed to collate and disseminate information about quality and academic 
standards, promote best practice and enhance quality. Each subject overview report includes a list of 
the participating universities in the subject area and the graded profiles achieved by these providers. 

Follow-up Reviews 

78. The prime purpose of the follow-up reviews is to evaluate the progress made since the subject 
review by measuring the impact of the review process on the quality of the subject and the academic 
standards set for, and achieved by, the graduates. The follow-up process is designed to support 
institutions in their development of internal systems and processes for sustainable improvement. The 
HFE is confident that all institutions will welcome the opportunity to demonstrate progress and the 
impact of its actions taken in response to the subject review outcomes and any other relevant 
developments. 

79. The timing of the follow-up review will be determined after discussions between the HFE and the 
institution. The review will normally be completed not sooner than two years and not later than five 
years after the date of the subject review site-visit. The timing will be influenced by three 
considerations: 

• the nature of the outcomes of the subject review as presented in the subject review 
report (that is, the time reasonably required for the institution to address any 
recommendations and demonstrate any impact of its actions) 



20 



Al Hussein Fund for Excellence: Subject Review Handbook January 2005 



• any significant changes to the circumstances in which the subject is organised and 
delivered (such as major rebuilding projects, major revisions to the curricula in 
response to other changes or a major restructuring of the organisation and delivery of 
the subject) 

• the degree of urgency and strategic importance attached by the HFE and the 
institution in evaluating the progress made and the impact of changes. 

80. The outcomes of the follow-up review require three sets of reference points: the six aspects with 
grade descriptors as set out above under The Subject Review Process and in annex L; the criteria on 
which the evaluation is based given in paragraph 81 below; and a set of measures or indicators used 
to demonstrate progress and impact as presented in paragraph 82. It is the responsibility of the 
institution to develop the measures and/or indicators and to assemble other evidence of progress and 
impact. 

81 . The criteria applied to the evaluation will be essentially the same as those used in subject review 
(please see annex L). However, in addition to the graded profile, the follow-up review will reach 
evidence-based conclusions on the progress made and the impact of recent actions in the institution. 
These additional criteria, contained in the aspect The Effectiveness of Quality Management and 
Enhancement, are that the institution: 

• Responds positively and effectively to external commentary, including from peer 
reviewers in the HFE subject review and other stakeholder groups by generating 
action plans that are focussed on key strategic matters, realistic, measurable and 
time-related 

• Is developing and applying appropriate internal systems and processes with clear 
aims, evaluation and reports that sustain and enhance the positive features of the 
recent subject review 

• Can provide evidence of performance and measures of impact of recent changes that 
give confidence in the capacity of the institution to assure and continually improve its 
quality and standards within the subject. 

82. The measures or indicators are determined by the institution. However, the HFE would not expect 
these to be less than the set of quantitative information presented in annex E, including a three-year 
series of student intake, progression and output data, and summary reports of student satisfaction 
surveys (preferably with "before and after" comparisons between the base-line and the most recent 
responses). The measures or indicators may, at the discretion of the institution, also address the 
question of the extent to which any targets for improvement were set and achieved, and any 
substantial raising of the academic standards represented in the programme specification(s). 

83. The base line for the measurement of progress will normally be the set of conclusions presented 
by the peer reviewers in the subject review report. However, when the institution has achieved a 
grade of 3 or higher for the effectiveness of Quality Management and Enhancement and the self- 
evaluation report (SER) was judged by the peer reviewers to be of a high standard, the base line for 
the follow-up review will be the SER. 

84. Documented evidence provided by the institution should include any management action report 
generated as a result of the subject review and a clear indication with examples and supporting 
evidence of the progress made and the impact of changes. The institution may present this 
information in one of the following forms: 

i. a recent internal review report or quality management report prepared for institutional 
purposes related to quality assurance and enhancement that meets the needs of the follow- 
up review 

ii. a collection of internal documents used as part of the normal internal action planning and 
reporting, suitably collated with a brief overview or explanatory note and an index 

iii. a brief SER written especially for the follow-up review not exceeding 5,000 words. 

85. The follow-up review will be scheduled by the HFE in discussions with the institution. The stages 
of follow-up review comprise: 
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• Preparation with the scheduling of the date for a site visit; the appointment of a review 
panel and the arrangements for the submission of advance documentation based on 
paragraph 84 above 

• A site visit normally lasting for two days concluding with an oral feedback report 

• A brief follow-up review report prepared by the review panel and submitted to the 
HFE. 

86. The follow-up review panel will consist of two external reviewers, of which one, if available, will 
have been a member of the subject review team, and the other will bring a fresh perspective. The 
panel will be led by one of the external members of the panel. For the follow-up review, the facilitator 
will if possible be the same person as made a contribution as a facilitator in the subject review. The 
facilitator role will be similar to that employed in the subject review (annex N). 

87. The agenda for the two-day site visit will be determined by the panel in consultation with the 
institution. The programme of activities, when agreed, should be copied to the HFE. It will invariably 
include at least one meeting with key staff directly involved in the action planning and delivery of any 
changes; a meeting with a representative group of students and a similar meeting with 
representatives of other key stakeholder groups such as employers of graduates and alumni. The 
panel will also wish to examine a sample of supporting evidence including revised documentation 
such as programme specification(s), samples of students' assessed work, together with analytical and 
evaluative reports and other internal management reports that demonstrate progress and impact. This 
evidence may include the reports to the institution of external examiners, verifiers or consultants 
engaged by the institution. 

88. The report prepared by the follow-up review panel will follow the basic format used in subject 
review with an introduction, an evaluation using the six aspects and a set of conclusions. The 
Effectiveness of Quality Management and Enhancement aspect will, in addition to the broad criteria 
set out in Annex A for subject review, address the three criteria set out in paragraph 81 above. The 
chairman of the follow-up review panel will submit the written report to the HFE. The HFE will wish to 
assure itself that the draft report meets the needs of the follow-up review in terms of reporting on 
progress and impact and addressing the criteria contained in paragraph 81 above. The HFE will 
make arrangements with the institution to forward a copy for the institution to comment on matters of 
factual accuracy before finalising the report and publishing it on the website. 

89. The HFE will draw upon follow-up review reports to compile an overall evaluation of the impact of 
subject review. This evaluation of the impact of subject review, which will also use additional 
information from its evaluation of the subject reviews and its analysis of the reported outcomes, will be 
published on its website. 
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Glossary 

Some of the terms appearing in the Subject Review Handbook and/or used in writing and using 
programme specifications in the higher education institution may have more than one meaning 
according to its context. To avoid confusion and to facilitate the understanding and the 
implementation of the subject review method, a brief glossary of terms is offered. 

Academic Standards 

Specific standards decided by the faculty or school, in line with its stated mission of the institution, 
that assure the standing of the academic award. The standards are derived from academic debate 
and custom, informed by the use of external reference points, including the attributes of the graduates 
expressed as the minimum knowledge and skills to be gained by the graduates from the academic 
programme. 

Accreditation: 

The recognition accorded by the Ministry to an institution which can demonstrate that its programs, or 
one or more designated programme of study, meet acceptable standards as specified by the 
Accreditation Requirements. It is recognized that institutions may also seek recognition of their 
programmes from other accrediting organizations particularly in professional and vocational 
disciplines. 

The impact of accreditation at programme level will be to require an assurance of the existence of a 
specific quality level in accordance with the institution's mission, the intended learning outcomes of 
the programme(s) and the expectations of similar academic institutions, the students and the labour 
market. 

Aims 

The aims are a collection of programme-specific goals, written in a general manner, expressing the 
broad purposes of the academic programme, concentrating on the knowledge, skills and attitudes that 
the programme intends to develop in the students. They are generally expressed in terms of "To 
provide..." or "To develop in students....". They should relate to the mission and also to the more 
detailed intended learning outcomes. They are not to be confused with strategic aims or goals (see 
below). 

Benchmarks: 

Reference points with which to compare the standards and quality of a programme. Reference points 
may be internal (for example, historic points of performance with which to compare current 
performance) and external (for example, the accreditation requirements set by the Ministry, or 
international statements of standards available to the institution - please see below). Some standards 
are expressed as inputs (for example the space required for private study in libraries). However, the 
development of academic standards worldwide places a greater emphasis on outcomes represented 
by the graduates. For example, subject benchmark statements such as those employed in the UK 
represent general expectations about the standards of achievement and general attributes to be 
expected of a graduate in a given subject area. Benchmarking refers to the process of using external 
reference points to articulate and verify that the academic standards are at least equivalent to those 
used in comparable institutions. 

Curriculum 

The curriculum is defined as the whole organized learning experience of the registered student. This 
includes the content (traditionally described in a written syllabus), but also addresses the organization 
of the range of learning opportunities provided within the educational programme. Where the 
programme includes capstone projects, internships or equivalent practical and applied learning, these 
are also regarded as part of the curriculum. 

External evaluator 

An external experienced person in the field of specialization who is invited to review the structure and 
content of a programme, its relevance to the ILOs, the standards and appropriateness of student 
assessments and attainment against the specification, and probably also evaluating the existing 
learning resources and whether or not they satisfy the programme requirements. The institution is 
responsible for specifying the evaluators' role and appointing them. 
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External Reference Points 

Points of reference used by an institution to establish, confirm or verify the academic standards for the 
educational programme(s). They may be other comparable institutions, existing accreditation 
requirements or benchmarks used by professional bodies and national and international agencies to 
specify standards. They are employed in institutions to assist it to verify the standards articulated in a 
programme specification and also to verify the actual level of achievements by graduates. 

Institution 

A university or equivalent educational institute that awards degrees and which contains faculties and 
schools that are responsible for designing and providing educational programmes operating within a 
set of policies, regulations and procedures. 

Intended Learning Outcomes (ILOs): 

The knowledge, understanding and skills which the institution intends the successful students to have 
gained on completion of the programme. For the purposes of quality assurance systems, these 
should be expressed as outcomes rather than as a traditionally written syllabus. They are the 
equivalent of "objectives" when this term is used to describe the intended level of achievement of the 
successful student. They are expressed at programme and course/module levels. 

Internal Systems and processes for Quality Management and Assurance 

The system adopted by the institution, and within the institution the faculty or school, to maintain and 
improve the quality of the educational programmes and other elements affecting them. Such an 
outcomes-related system involves precise specifications for quality, the identification of good practice 
as well as of learning deficiencies and obstacles, performance follow-up, suggestions for development 
and enhancement, and the systematic review and development of processes for establishing effective 
policies, strategies and priorities to support continuing improvement. It will be underpinned by formal 
policies, regulations and procedures and the means of accounting for performance. 

Learning Resources 

The facilities accessible by registered students and their tutors that support the curriculum. These 
generally include accommodation, libraries including catalogue, web services and librarian together 
with their stock of texts, journals and other materials, computing and other IT facilities, laboratories 
and workshop facilities and specialist equipment. They may also be defined to include additional 
facilities outwith the school, faculty or institution such as the resources organized in professional and 
vocational practice for internships. For the purpose of subject review the learning resources aspect 
does include technical and administrative support staff, but excludes the match of academic staff to 
the curriculum, which is addressed under Curriculum Design, Content and Organisation. 

Peer Reviewer 

A person who is professionally equal in calibre and subject specialism to those delivering the 
provision but not from the same institution or have any other conflict of interest, who can contribute to 
the review of an educational programme both for external reviews arranged by the HFE and for formal 
accreditation review purposes. 

Programme evaluation 

The process used to evaluate or review the educational programme and to obtain the opinions of the 
stakeholders of the programme, including students, faculty members, the graduates, and the 
governing council with the aim of improving and developing the programme to cope with the advances 
in subject matter and the needs of society and the environment. Self-evaluation is central to internal 
quality assurance systems. External reviews, for example in connection with an external subject 
review, will use the self-evaluation reports as a starting point and lead to an evaluation report. 

Programme Specification 

A programme specification is a concise description of the intended outcomes of learning from a higher 
education programme, and the means by which these outcomes are achieved and demonstrated. It is 
intended for use by all those directly involved in the design and delivery of the programme, including 
students, staff, peer reviewers and external evaluators, and should also be of interest to other 
stakeholders such as the employing community. 
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Quality Assurance 

For the purposes of this process, quality assurance is defined as the systems and processes applied 
by a responsible organisation to ensure that, informed by its mission, academic standards are defined 
and achieved in line with equivalent standards nationally and internationally, and that the quality of 
learning opportunities, research and community involvement are appropriate and fulfill the 
expectations of the range of stakeholders. 

Stakeholders: 

Those groups and individuals which have a legitimate interest in the educational activities of the 
institution both in respect of the quality and standards of the education and also in respect of the 
effectiveness of the systems and processes for assuring the quality. An effective internal review 
process or self-evaluation will include the key stakeholder groups. The precise range of stakeholder 
groups and their differentiated interests depend upon the mission, the range of educational activities 
in the institution and local circumstances. The range is usually defined by a scoping study. Examples 
of groups with a legitimate interest include current students, alumni, intending students, staff in the 
institution, the employing community, the Ministry of Higher Education and Scientific Research, Non- 
Governmental Organisations (NGO), the sponsors and other funding organisations and, where 
appropriate, professional organisations. 

Strategic Aims or Goals 

A collection of aims that are specific to the institution, school, faculty, department or educational 
programme that set out the means to achieve a mission or an explicit development project for 
improvement. They are usually derived from an assessment of needs and take account of the 
strengths, weaknesses, threats and opportunities, the preferred direction and the chosen priorities. 
They are written in a general manner, focusing on the academic programmes with due regard to 
interests of the clients and other stakeholders. They are likely to take a medium and long-term view. 

Student Assessment: 

The different types of assessment including examinations or semester activities that the teacher sets 
to ensure that the students have achieved the ILOs. Diagnostic assessment provides an indicator of a 
learner's aptitude and preparedness for a programme of study and identifies any specific individual 
learning needs or interests. Formative assessment is regarded as part of the process of learning and 
requires a high degree of dialogue between the teacher and the student to provide the students with 
feedback on achievement and progress and help them determine further learning that will enhance 
performance and understanding. Summative assessment is generally organised at the end of a unit, 
course or module of a programme. Assessment methods range widely from formal, time-constrained, 
unseen, closed-book examination at the end of a course or programme to projects that produce an 
artifact, a dissertation or a portfolio of work. 

Subject 

The cognate area addressed by an educational program. It may include one or more disciplines within 
it, particularly in vocational and professional subjects where multi-disciplinary and inter-disciplinary 
study are common. For example, "Engineering" typically includes disciplines such as Civil 
Engineering, Mechanical Engineering and Electrical and Electronic Engineering. Educational 
programmes may also address multi-disciplinary areas such Production Engineering as well as 
contain essential elements such as applied mathematics. 

Teaching, Learning and Assessment: 

The methods and processes used to help students to achieve the ILOs for the course. The range of 
methods will generally be informed by a strategy for learning, teaching and assessment. Examples of 
methods include: a case study to teach students how to analyse information and reach a decision 
based on available evidence; writing a review paper for the students to gain the skills of self-learning 
and presentation; practical sessions for the students to gain practical skills; distance learning with a 
multi-media range of materials; and executing laboratory or field-work experiments to train the 
students to gather information, analyse the results and reach specific conclusions. Independent 
learning frequently features in current curricula and assessment methods, or at least in the strategic 
aims of institutions. It may include placing varying degrees of responsibility for learning upon the 
students, ranging from a strategic goal to produce graduates who are autonomous self-sustaining 
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learners in their intended professions, to a negotiated curriculum to the setting of personal targets and 
the recorded acquisition of personal skills in a portfolio. 



Follow-up Review 

Follow-up reviews extend the subject review process and provide the institution with an opportunity to 
demonstrate the progress made and the impact of actions taken after the subject review site visit. 
They are designed to support the development of internal systems and processes for sustained 
quality assurance and continuing improvement. 
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Annex A 

The Subject Review Aide Memoire 
Introduction 

1. These notes are structured around the six aspects of provision, and offer a set of prompts which 
are designed to ensure focused coverage of each aspect of provision throughout the review and the 
associated report writing. 

2. This aide memoire is designed for use by institutions undertaking self-evaluation, and by reviewers 
to guide the: 

• analysis of the self-evaluation report (SER) prior to the visit 

• gathering of evidence during the visit 

• writing of the subject review report. 

3. The key features for each aspect of provision are covered broadly by this aide memoire. However, 
the content of the aide memoire should not be regarded as prescriptive or exhaustive. The stated 
aims of the subject provider together with the programme specification(s) and the self-evaluation 
report form the basis for review. Reviewers use the aide memoire together with the SER, their 
analysis of it, the preparatory e-mail discussions between the review chair and the institution's 
representatives and other documentation, to add and/or highlight questions and issues which have 
been identified as being of significance within the particular provision being reviewed. 

4. It is essential that institutions embarking on self-evaluation, and visiting peer reviewers, maintain 
written records of evidence as it is gathered, including notes of meetings and sources in written 
documentation, as well as completed observation notes and student work/assessment notes. As a 
general principle, for subject review, institutions and reviewers should focus their attention at the level 
of the subject and address central institutional issues only when these have a direct impact on the 
subject being reviewed. 

5. In subject review, the aide-memoire should be applied to address generically the range of 
provision. When the HFE has invited the institution to put forward for external review more than one 
programme, the aide-memoire should be used to address the range and only discuss individual 
programmes when there is a need to differentiate. 
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The Subject Review Aide Memoire: 
Curriculum Design, Content and Organisation 

Consider the programme specification(s), and identify the relevant aims and 
ILOs, and statements in the self-evaluation report (SER). 



Evaluate the extent to which this aspect contributes to meeting the programme 
specification(s) with particular reference to the ILOs. Do the ILOs set, and the 
curricula informed by those ILOs, allow the aims to be met? 

1. Evaluation of the intended learning outcomes in relation to the broad aims 
of the provision and to any applicable external reference points. 

Reviewers should ask: 

What are the intended learning outcomes (ILOs) for the programme(s)? 

How do they relate to the overall aims of the programme(s) as stated by the institution? 

Are they appropriate to the aims? 

How do they relate to external reference points including, where applicable, relevant benchmark 
statements and accreditation requirements by the Ministry and professional organisations? 

Potential sources of information and evidence will include the self-evaluation, programme 
specification(s), the most recent annual reports, curricular documents, subject benchmark statements, 
the Ministry's accreditation requirements and details of requirements of professional organisations. 

Review activities may also include an analysis of programme content, discussions with members of 
the teaching staff. They should then evaluate the intended learning outcomes against the aims of the 
provision as described in the self-evaluation and against relevant external reference points. As a 
result of these activities reviewers should be able to judge: 

• whether the ILOs are clearly stated; 

• whether they reflect appropriately the overall aims of the provision and relevant 
benchmark statements and other external references. 

2. Evaluation of the effectiveness of the means by which the subject provider 
designs and organises the curriculum to support the attainment of the 
intended learning outcomes. 

Reviewers should ask: 

How does the institution ensure that curriculum content enables students to achieve the intended 
learning outcomes? 

How does the institution ensure that the design and organisation of the curricula are effective in 
promoting student learning and achievement of the intended learning outcomes? 

Sources of information and evidence will include the SER, curricular documents such as manuals 
or handbooks, and curricular review and validation reports. Reviewers should seek information about 
levels and modes of study, breadth and depth of study, inter- and multi-disciplinarily, coherence, 
flexibility and student choice, as well as the role of professional and/or statutory bodies where 
relevant. 

Review activities may also include discussions with members of the teaching teams, support staff 
and administrative staff, and discussions with students. They should then evaluate the effectiveness 
of the way in which the institution plans, designs and approves the curricula. As a result of these 
activities reviewers should be able to assess the adequacy of procedures for ensuring that 
programmes are designed to enable students to achieve the ILOs. 
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3. Evaluation of the means by which the intended learning outcomes are 
communicated to students, staff and other with a direct interest such as 
potential employers. 

Reviewers should ask: 

How are the ILOs of a programme and its constituent parts communicated to staff, students and 
external examiners? 

Do the students know what is expected of them? 

Sources of information and evidence will include the SER, programme specifications, programme 
or subject handbooks and documents such as module guides and assessment schedules. 

Review activities may also include discussions with teaching teams and students. They should then 
evaluate the way in which subject providers convey their expectations to staff and students. As a 
result of these activities reviewers should be able to judge the adequacy of arrangements within the 
subject for communicating intended learning outcomes. 

4. Evaluation of the means by which the institution creates the conditions for 
the attainment of the intended learning outcomes. 

Reviewers should ask: For each programme being reviewed, do the design, content and 
organisation of the curriculum support the attainment of the ILOs in terms of knowledge, cognitive 
skills, subject specific skills (including practical/professional skills), key transferable skills, personal 
development including independent learning and progression to employment and/or further study? 

Is the collective expertise of the academic staff suitable and available for effective delivery of the 
curricula and for the achievement of the ILOs? 

Sources of information and evidence will include the SER, subject or programme handbooks and 
curricular documents, such as course or module handbooks, notes of guidance on practicals, projects 
and placements, staff CVs and further study and employment statistics. 

Review activities will also include evaluation of curricular documents and discussions with staff and 
students. They should then evaluate the design, content and organisation of the curriculum for the 
programme(s) in relation to the potential for enabling students to achieve the ILOs. As a result of 
these activities reviewers should be able to judge whether the intended learning outcomes are 
adequately supported by the curriculum and the staff profile, and that the curriculum maintains 
relevance. 

5. Evaluation of the breadth, depth and currency of the curricula 

Reviewers should ask : Is the curriculum content appropriate, in breadth and depth, to each stage of 
the programme(s), and to the level of the award(s) and does it secure academic and intellectual 
progression by imposing increasing demands on the learner, over time, in terms of the acquisition of 
knowledge and skills, the capacity for conceptualisation, and increasing autonomy in learning? 

And also : Is there evidence that curricular content, design and organisation is informed by current 
research and other scholarly activity, by up-to-date assessments of the labour market, by any 
changes in relevant occupational or professional requirements and by recent developments in 
approaches to teaching and learning in the discipline? 

Sources of information and evidence will include the SER, programme specification(s), subject or 
programme handbooks, validation or accreditation documents, and professional and/or statutory body 
accreditation reports. 

Review activities may also include discussions with staff, discussions with professional and/or 
statutory bodies, and discussions with employers (where relevant and possible). They should then 
evaluate whether the curricula are appropriate in these respects and whether they are adequately 
informed by the wider range of academic activities and recent developments in teaching and learning 
approaches. As a result of these activities reviewers should be able to assess the breadth, depth, and 
currency of the curricula. 



29 



Al Hussein Fund for Excellence: Subject Review Handbook January 2005 



The Subject Review Aide Memoire: 
Teaching, Learning and Assessment 

Consider the programme specification(s), and identify the relevant aims and 
ILOs, and statements in the self-evaluation report. 

Evaluate the extent to which this aspect contributes to meeting the programme 
specification(s) with particular reference to the ILOs. Do the ILOs, and the level 
of attainment of those ILOs through the teaching learning and assessment 
activities, allow the aims to be met? 

1. Evaluation of the organisation of teaching, learning and assessment 
Reviewers should ask: 

What is the strategy for teaching, learning and assessment? Does it articulate clearly with the 
programme specification(s) and support the attainment of the ILOs? 

Are the programmes of teaching, learning and assessment activities appropriate in terms of the ILOs, 
in particular the development and assessment of: 

• knowledge and understanding 

• cognitive skills 

• subject-specific skills, including practical/professional skills 

• key (transferable) skills 

• progression to employment and further study 

• personal development including independent learning 

Sources of information and evidence includes the SER, internal and external reports, teaching 

and learning materials, student evaluation questionnaires and meetings with staff, alumni and 

students, student handbooks, timetables and programmes of teaching and policy documents. 

Review activities will invariably the scrutiny of students' work and may also include class 
observations. The reviewers can then form a judgement on the extent to which there is a coherent 
approach to teaching, learning and assessment that effectively addresses the ILOs and meets the 
stated aims. 

2. Evaluation of the quality of the teaching by staff and how it supports 
learning. 

Reviewers should ask: 

How effective is teaching in relation to curricula content and the programme specification(s)? 

How effectively do staff draw upon their research, scholarship or professional activity to inform their 
teaching? 

How good are the materials, including e-learning, provided to support learning? 
Is there effective engagement with and participation by students? 

Is the quality of teaching maintained and enhanced through effective staff development, peer review 
of teaching, integration of part-time and visiting staff, effective team teaching and induction and 
mentoring of new staff? 

How effectively is learning facilitated in terms of student workloads? 

Sources of information and evidence will include the SER, student questionnaires, internal review 
documents, staff development documents, subject or programme handbooks, and academic staff 
appointment documents such as CVs. 
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Review activities may also include direct observation of teaching and practical workshop sessions, 
discussions with staff, and discussions with students. As a result of these activities reviewers should 
be able to make an overall judgement about the quality of teaching and learning opportunities and the 
extent to which teaching and learning contribute to the achievement of the ILOs. They should then 
evaluate the overall effectiveness of the teaching and learning activities, including: 

• the breadth, depth, pace and challenge of teaching; 

• whether there is suitable variety of appropriate teaching methods; 

• whether there are suitable opportunities for guided and independent learning by 
students; 

• the effectiveness of the teaching and learning of subject knowledge; and 

• the effectiveness of the learning of subject-specific, transferable and practical skills. 

3. Evaluation of the assessment process and the academic standards it 
demonstrates 

Reviewers should ask: 

Does the assessment process enable learners to demonstrate achievement of the ILOs? 

Are there criteria that enable teachers, students, internal and external examiners to distinguish 

between different levels of achievement? 

Does the assessment strategy have an adequate formative function in promoting student learning, 
and how (e.g. feedback, further reading tasks)? 

Can there be full confidence in the objectivity, fairness and transparency of assessment procedures? 

Sources of information and evidence will include the SER and programme specification(s), 
assessment criteria and guidance to markers, external examiners' reports and procedures for 
monitoring and recording achievement. 

Review activities may also include discussions with teaching teams, students and external 
examiners and the analysis of the methods for recording progress and achievement. They should 
then evaluate whether the overall assessment process and the particular assessment methods 
chosen are appropriate and effective. As a result of these activities, reviewers should be able to judge 
whether assessment processes can adequately measure achievement of the ILOs for the programme 
and that the process gives confidence in the academic standards attained. 
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The Subject Review Aide Memoire: 
Student Progression and Achievement 

Consider the programme specification, and identify the relevant aims and 
ILOs, and statements in the self-evaluation report. 

Evaluate the extent to which this aspect contributes to meeting the programme 
specification(s). Do the ILOs, and the level of attainment of those ILOs, allow 
the aims to be met? 

1. Evaluation of profile of students 
Reviewers should ask: 

Does the profile of applications and admissions match the mission and the stated aims in terms of: 

• ratio of applications to places 

• entry qualifications 

• any special features related to the mission, such as profile at entry in relation to the labour 
market, and regional cross-border recruitment? 

Are the rates and trends in student progression and completion satisfactory in terms of: 

• progression and non-progression at each stage of the programme (differentiate failure, length 
of study, withdrawal and transfers in and out) 

• completion of the programme 

• qualifications awarded (for example, degree results)? 

Sources of information and evidence include data on student admissions, induction and 
progression rates, perhaps with the outcomes of diagnostic assessment. There may be contextual 
evidence of analyses of trends in the labour market and patterns of recruitment in the region. Staff 
and students will also provide useful comment on expectations and the quality of information 
supporting decisions on admission, registration and progression through the programme at each 
level. 

Review activities w ill include meetings with staff, students and representatives of the employers as 
well as scrutiny of documentation and data sets. As a result of these activities, reviewers should be 
able to judge the appropriateness of the profile at entry and in progressive stages through the 
programme in relation to the programme specification(s) and in particular to the ILOs. 



2. Evaluation of student achievement at appropriate academic standards. 
Reviewers should ask: 

What evidence is there that the standards achieved by students meet the minimum expectations for 
the award, as measured against the ILOs, relevant reference standards and any other applicable 
requirements? 

Does the range of students' levels of attainment, given the entry profile, confirm that an appropriate 

proportion of able students demonstrate exceptional achievement? 

How is student achievement, including progression to employment/further study, verified? 

Sources of information and evidence will include the SER, programme specification(s), 
external verifiers' or examiners' reports, other reports of professional/validating bodies, where 
appropriate 

employers/professional practitioners, examination board minutes, records of pass rates, the 
analyses of destination data and satisfaction surveys, and samples of student work. Relevant 
reference standards will be an important point of reference. 

Review activities may also include discussions with teaching teams and the programme 
leader/group and the examiners. They should then evaluate whether student achievement meets or 
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exceeds the minimum expectations. As a result of these activities, reviewers should be able to judge 
whether appropriate standards are being marked, achieved and verified. 
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The Subject Review Aide Memoire: 
Student Support and Guidance 

Consider the programme specification(s), and identify the relevant aims and 
ILOs, and statements in the self-evaluation report. 

Evaluate the extent to which this aspect contributes to meeting the ILOs set. 
Do the ILOs set, and the level of attainment of those ILOs through the 
arrangements for support and guidance, allow the aims to be met? 

Evaluation of the quality of support and guidance 
Reviewers should ask: 

Is there an appropriate overall strategy for academic support, including written guidance, which is 
consistent with the student profile and the overall aims of the provision? 

Is there clear written guidance relating to the programme(s) and the support and guidance available? 
Are students' needs identified and addressed appropriately through practical arrangements for 
students' access to tutors and the systematic monitoring of students' progress (SPA)? 
Are arrangements made to identify and support students with special learning needs, including 
outstanding students? 

If the language of teaching is not Arabic, do students have an adequate level of skills in the 

language? Do students receive adequate foreign language support to support their learning (for 

example, to enable them to access up-to-date textbooks, technical literature, the Internet)? 

How effectively is learning facilitated by academic guidance, feedback and supervisory 

arrangements? 

Are students studying off-site or on internships receiving academic support? 

Are the arrangements for academic tutorial support clear, effective and generally understood by staff 
and students? 

Are there appropriate arrangements for non-academic support such as personal and welfare 
considerations? 

Are the careers information and guidance well matched to the ILOs and the labour market? 

Sources of information and evidence will include subject or programme handbooks, student 
questionnaires, internal review documents, recruitment data, and progression data. 

Review activities may also include discussions with admissions staff, discussions with teaching staff 
and tutors, and discussions with students. They should then evaluate whether the arrangements in 
place are effective in supporting students and providing appropriate guidance to successful 
completion of their programmes. As a result of these activities, reviewers should be able to judge the 
effectiveness of the student support and guidance, the strategy for student support and the impact 
they have on the attainment of the ILOs and the stated aims. 
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The Subject Review Aide Memoire: 
Learning Resources 

Consider the programme specification, and identify the relevant aims and 
ILOs, and statements in the self-evaluation report. 

Establish student views on learning resources and, where possible, the extent 
of use by students in the subject. Evidence should be derived in part from 
completed notes on class observations and students' assessed work (Annexes 
H and I). 

Evaluate the extent to which this aspect contributes to meeting the programme 
specification(s). Do the ILOs, and the level of attainment of those ILOs, 
through the use of available learning resources, allow the aims to be met? 

Evaluation of the quality of learning resources and their deployment. 
Reviewers should ask: 

Is there an overall strategy for the deployment and continuing enhancement of learning resources? 

How effectively is learning facilitated in terms of the provision of physical resources? 

Is suitable teaching and learning accommodation available? 

Are the subject book and periodical stocks appropriate and accessible? 

Are suitable equipment and appropriate IT facilities available to learners, including internet and 
intranet access? 

Do students make full use of the available facilities? 

Is appropriate technical and administrative support available? 

Sources of information and evidence will include equipment lists, library stocks, and internal review 
documents including internal reports. Where class observations take place, these can be valuable 
source of information. Discussions with staff and students 

Review activities may also include direct observation of accommodation and equipment, discussions 
with staff, and discussions with students. They should then evaluate the appropriateness of the 
physical learning resources available and the effectiveness of their deployment. As a result of these 
activities, reviewers should be able to judge how effectively the available learning resources are 
deployed in support of the ILOs. 
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The Subject Review Aide Memoire: 

The Effectiveness of Quality Management and Enhancement 

Consider the programme specification(s), and identify the relevant aims and 
ILOs, and statements in the self-evaluation report. 

Evaluate the extent to which the systems and processes in place maintain and improve the quality 
and academic standards of the programme(s) and promote a level of confidence in the academic 
standards of the academic award(s). 

Evaluate the extent to which this aspect contributes to the development of the programme 
specification(s) and assure the achievement of the ILOs . Do the ILOs set, and the levels of 
attainment of those objectives through the effectiveness of quality management and 
enhancement, allow the aims to be met? 

Evaluation of the effectiveness of the arrangements for quality management 
and enhancement 

Reviewers should ask: 

How effective are the internal arrangements for specifying, monitoring and evaluating the 
programme(s) provided? 

Do these arrangements involve appropriate consideration of: 

• national policies and priorities 

• the mission of the institution 

• the policies, regulations and procedures of the institution 

• the use of management information such as internal monitoring data (for example, on the 
quality of teaching and learning, student satisfaction surveys, student progression and 
achievement, learning and other support services) 

• the views of staff 

• the views of students 

• the views of other key stakeholders including employers and professional bodies where 
appropriate 

• other internal or external reviews and reports 

• professional development needs? 

How effective are the processes of self-evaluation and continuing improvement, including: 

• the clarity of the programme specification(s) and in particular the ILOs 

• the quality of the self-evaluation process and the report 

• engagements with the range of key stakeholder groups? 

• appraisal and internal peer review of teaching 

• a focussed professional development programme 

• the contributions of staff in the range of activities such as professional development, 
scholarship and academic research, community involvement and projects to enhance the 
provision? 

What impact is evident of the quality management and enhancement systems and processes in terms 
of: 

• the use of external reference points to establish and verify academic standards 

• the development and application of internal indicators/measures of performance 

• the quality of the learning opportunities 

• the level of achievement of graduates 

• improvement arising from earlier action plans 

• continuing review and revision of the programme specification(s) 
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• taking a strategic view of future plans for enhancement 

• the generation and use of internal reports 

• the identification and dissemination of good practice 

• the identification and implementation of action points required to meet the programme 
specification more fully and in the spirit of continuing improvement? 

Sources of information and evidence will include the annual self-evaluation, other internal and 
external review documents, policy statements and programmes of professional development (of staff) 
including any measures of impact, records and analysis of statistical data, arrangements for 
representation of students, use made of qualitative feedback from students and staff, summary 
reports of recent surveys, external examiners' reports, professional and/or statutory body 
accreditation reports, examination board and other committee minutes. 

Review activities will include an evaluation of the adequacy of the mechanisms and procedures used 
and the effectiveness of their implementation by the institution. Reviewers may also include analyses 
of information, practices and procedures, discussions with teaching teams and discussions with a 
representative group of graduates, employers and others with an interest. 

As a result of these activities reviewers should be able to assess the capacity of the subject provider 
to apply the institution's policies, regulations and procedures in line with the mission; maintain and 
enhance its system and processes of quality assurance; and maintain and enhance the 
programme(s). 
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Annex B: Guidelines and Template for Programme Specifications 
Introduction 

1 . The purpose of these guidelines is to assist higher education institutions to prepare a specification 
for each academic programme in their institution. In the longer term the same structure may be 
developed to prepare course or module specifications and (with the addition of performance indicators 
and action planning) to write annual reports on the operation of the programme(s) and its courses. 
These guidelines draw on good quality assurance practice nationally and internationally in institutions 
of higher education. The notes and the recommended template for programme specification are 
developments of documents published by the Quality Assurance Agency for Higher Education in the 
UK. 

2. Programme specifications are an important tool in assuring the quality of education. The 
programme specification template (attached as Annex B1), reflecting this quality assurance function, 
contains the following main items: basic information about the programme; the educational aims; the 
intended learning outcomes (ILOs); the external reference points used to confirm the relevance and 
currency of the ILOs and their equivalence to comparable degree programmes (sometimes referred to 
as benchmarks); curriculum structure and content; courses (modules) in the programme; a matrix of 
the skills developed across the various components of the programme; arrangements for student 
assessment. 

3. They should be read and used in conjunction with Annex A (the Subject Review Aide-Memoire) 
and Annex C (Guidelines on Self-Evaluation). 

4. The Programme Specification template is given in Annex B1. In addition, a draft template for 
course specification to accompany the programme specification is provided in Annex B2. This is 
optional for institutions wishing to develop a comprehensive system. 

The function of a programme specification 

5. In order to assure the production of graduates according to internationally recognized standards, all 
institutions need to develop appropriate internal systems to specify the quality of their academic 
programmes, specify the intended outcomes in terms of the attributes of their graduates, to review 
and report routinely on their performance against the specification and to take steps to identify 
matters that need to be addressed as part of a process of continuing improvement. Such action will 
increase the skills of graduates and enhance their competitive capacity in the national and regional 
labour market. Hence, it is important to specify each programme according to national and 
international standards and on the basis of its intended learning outcomes (ILOs). It is also important 
to prepare an annual report on the performance of the programme during its implementation. This 
must be done with reference to the standards and benchmarks that are to be carefully chosen by the 
HE institution in accordance with its mission. 

A Guide for the Preparation of Programme Specifications 

6. This guide is intended to facilitate a programme team, the school or faculty and the institution, in 
the development of suitable programme specifications. 

7. All programmes being reviewed by arrangement with the HFE will be required to complete a 
Programme Specification. 

8. The template is based on those used in UK universities. 
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What is a Programme Specification? 

'A programme specification is a concise description of the intended outcomes of learning from 
a higher education programme, and the means by which these outcomes are achieved and 
demonstrated' 

What is the Purpose of a Programme Specification? 

A programme specification should facilitate the reader (staff, students and others e.g. 
employers) to identify: 



A. The intended learning outcomes (ILOs) of the programme in terms of: 



• Knowledge and understanding that a student will be expected to have upon 
completion 

• Key skills: communication, numeracy, the use of IT and learning how to learn 

• Cognitive skills, such as understanding of methodologies or ability in critical analysis 

• Subject specific skills 



B. Information to assist students in their understanding of the: 



• teaching & learning methods used to enable outcomes to be achieved 

• assessment methods used to enable outcomes to be demonstrated 

• relationship of the programme and its study elements to 

- the degree awarded 

- any subsequent professional qualification 
-any career path 

To Write a Programme Specification 

• Attend a workshop provided by the HFE or the institution 

• Use an active programme team or form a working group 

• Use the template provided 

• Refer to existing written information on the programme including the accreditation 
requirements and any definitive programme description 

• Consult with key members of the institution such as a quality unit or centre and 
representatives of key stakeholder groups 

• Read pages 3-5 in the QAA Guidelines for preparing programme specifications- 
website address below 

• Look at examples in the institution and in the above QAA Guidelines 



Guidelines for preparing programme specifications used in the UK may be found at 
http://www.qaa.ac.uk/crntwork/progspec/progspec0600.pdf 
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Creating a programme specification 

What follows is a suggested series of Stages and Tasks which a programme team working in groups 
may wish to follow. 



STAGE ONE: 



1 . What is it about this programme that we want our students to achieve? 

This may include ... 

• subject knowledge and understanding 

• a range of intellectual skills, subject specific skills, transferable (key) skills, aesthetic 
skills etc. etc. 

• application of these skills in a range of contexts 

Task one: Create some programme outcome statements about our programme, by completing 
the following sentences 

• This programme is distinctive because it develops . . . 

• The most important values, which inform this programme, are. . . 

• The academic content of this programme concentrates on. . . 

• The most important intellectual skills developed in this programme are. . . 

• The most useful practical skills, techniques and capabilities developed are. . . 

• Competency will be developed in. . . 

• The most important ways a student will learn are. . . 

• On completing the programme, we want students to know and understand. . . 

• On completing the programme, we want students to be able to. . . 

STAGE TWO: 



2. What reference points can we use to show that which we want students to achieve has 
relevance and currency within the academic, professional and employing communities? 

Task two: Select the relevant reference points and decide how each might be used 

• University mission statement 

• Measures and Criteria for Accreditation in Jordan 

• Requirements of professional bodies in Jordan or internationally 

• Others not listed above but relevant to the programme 

STAGE THREE: 



3. Do we know/agree on what it is we want our students to achieve as a result of studying 
this programme? What are the aims and the intended learning outcomes (ILOs) of the 
programme as a whole? 
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Task three - using issues identified in Tasks 1 and 2: 
3.1 : what do you consider to be the: 

• aims of the programme 

• ILOs of the programme? 



3.2: Recognise as a whole group that the programme learning outcomes will generally be 
achieved through the learning outcomes specified for individual modules or courses. 



3.3: Consider the design of the modules or courses which make up the programme, bearing in mind 
that the overall outcomes of the programme also take place in the context of the level or stage of 
learning that students are at. 



3.4: Identify: 

• Where these outcomes will be found in the programme 

• Where these outcomes are taught/learnt in the programme 

• Where these outcomes will be assessed in the programme 

• Consider also how these outcomes match your chosen reference points e.g. 
Benchmarks/ institutional policies and mission e.g. employability skills & curriculum 

• Consider how these outcomes are distributed across the modules and the various 
levels/stages of the programme. 



STAGE FOUR: 

4. How would these outcomes be best/most appropriately taught and assessed? 

Task Four: 

Consider the teaching, study and assessment methods used to promote learning. Some methods are 
more appropriate than others for developing particular types of learning. Examples are given below. 
Identify existing good practice in the programme and any weaknesses that should be addressed as 
part of the development of the specification. 

• Knowledge and understanding of a subject is often developed through lectures and 
seminars. Such direct teaching methods are usually supported by directed study of textbooks 
and journal articles (hard copy or electronic), wider reading instigated by the students and by 
assignment or project work. Knowledge and understanding of a subject is often assessed 
through unseen written examinations, but most if not all assessment methods will require some 
demonstration of knowledge and understanding 

• Intellectual skills such as analysis, synthesis, evaluation, problem solving may be practised 
and demonstrated through more active learning processes involving assignments or projects, 
group-learning activity such as a seminar or tutorial, laboratory, workshop or field based activity. 
Assessment of intellectual skills can utilise unseen written examinations or problem based 
exercises. Independent project work or research dissertations are typically used to demonstrate 
capability in a range of intellectual skills linked to the development and application of specialist 
knowledge, understanding and practical skills. 

• Practical skills need to be developed through opportunities to practise the activity in an 
appropriate learning context (e.g. in laboratory, field, or workplace placement). Workbooks or 
guidance manuals may be used to support learning. Assessment of competence in exercising a 
practical skill must involve practical demonstration of it. 
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• Transferable/ key skills, that are readily transferable to employment and other contexts, 
such as communication/teamwork can be developed through naturally arising opportunities in the 
curriculum. For example, written communication skills can be developed and assessed through 
essays or dissertations; oral communication skills through presentations in seminars; or team- 
working skills through collaborative projects. Skills may be developed through extra-curricular 
activities including work experience, student representative work, and social and cultural 
activities. 

STAGE FIVE: 



5. Complete the template 



Task five: 

Use the template to compile and present the whole specification. 



STAGE SIX: 



6. Create or develop existing course specifications using the programme specification 
template and exiting documentation. Ensure that all of the programme's ILOs are covered. 



Task six: 

For each course prepare a course specification that represents the contribution the course makes 
to the programme. Refer to sections 13 and 18 of the programme specification template to confirm 
the contribution each course makes. You may wish to develop a template or adopt one based on 
the B2 template provided here to record the course specifications in a standard format. 



STAGE SEVEN: 

7. How is the programme specification to be communicated effectively to all with a legitimate 
interest? 

Task seven: 

7.1: Review existing programme documentation and website information and clarify the purposes of 
programme specifications within the institution. 

7.2: Develop a strategy for phased implementation if the development of the programme specification 
has led to material change in the curriculum, such as adjusting the range of student assessment 
methods. 

7.3: Consider the most appropriate form of presentation to the audiences (e.g. staff, visiting 
professors, students, external evaluators) and prepare a format (e.g. web page or intranet site) to 
serve the purposes. 

7.4: Test the draft for accuracy, consistency and clarity of content and presentation and its 
accessibility, and then 
7.5: Implement. 

7.6: Include a review of the programme specification in the next regular internal review to fine tune it 
and learn from the experience. 
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Annex B1 : Programme Specification Template 



UNIVERSITY OF 



Programme Specification 



This Programme Specification provides a concise summary of the main features of the programme 
and the learning outcomes that a typical student might reasonably be expected to achieve and 
demonstrate if he/she takes full advantage of the learning opportunities that are provided. 



1. Awarding Institution / Body 




2. Teaching Institution (if different) 




3. School/Faculty and 
Department/Centre 




4. Title of Final Award 




5. Modes of Attendance offered 




6. External Accreditation and 
date(s) 




4. Other external references 
used to verify standards (a) 
at design stage and (b) at 
graduation (achievement) 




5. Date of production/revision 
of this template 




6. Aims of the Programme 
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7. Intended Learning Outcomes (ILOs), Teaching, Learning and Assessment 
Methods 

A. ILOs for Knowledge and Understanding 

e.g. 
A1. 
A2. 
A3. 

A4. 

Teaching and Learning Methods 



Assessment methods 



B. ILOs for subject-specific skills 

e.g. 
B1. 
B2. 
B3. 
B4. 



Assessment methods 



C. ILOs for thinking skills 



e.g. 
C1. 
C2. 
C3. 
C4. 



Teaching and Learning Methods 



e.g. 
D1. 
D2. 
D3. 
D4. 



Teaching and Learning Methods 



Assessment methods 
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11. Programme Structures* 


12. Awards and Credits* 


Level 


Module 
Code 


Module Title 


Credit 
rating 




Level 4 








Masters Degree 

Requires xxx credits at 
Level 3 or above with a 
minimum of xxx credits at 
Level 4 


Level 3 








Bachelor Honours 
Degree 

Requires xxx credits 
including a minimum of 
xxx at Level 2 and xxx at 
Level 3 

Bachelor Degree 

Requires xxx credits 
including a minimum of 
xxx at Level 2 and xxx at 
Level 3 


Level 2 








HE Diploma 

Requires xxx credits 
including a minimum of 
xxx at Level 2 


Level 1 








HE Certificate 

Requires xxx credits 


Levels 

A/B/C 

(FE) 








Certificate of 
Achievement 

Requires a minimum of 20 
credits 



* Delete rows not applicable to this Programme Specification 



13. Personal Development Planning 



14. Admissions criteria 
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15. Key sources of information about the programme 
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16. Curriculum Skills Map 



Please tick in the relevant boxes where individual P rogramme Learning Outcomes are being asst 





Programme Learning ( 


Dutcomes 


Level 


Code 


Module/Course Title 


oore 
(0) 


Knowledge and 
understanding 


Subject-specific 
Skills 


T 

1 


A1 


A2 


A3 


A4 


B1 


B2 


B3 


B4 


c 


e.g. LEVEL 3 


















































































































































e.g. LEVEL 2 


















































































































































e.g. LEVEL 1 



















































































































































Note: Mapping to other external frameworks, e.g. professional/statutory bodies, will 
be included within Student Handbooks 
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Annex B2: Course Specification Template 



UNIVERSITY OF 



Course Specification 



This Course Specification provides a concise summary of the main features of the 
course or module and the learning outcomes that a typical student might reasonably 
be expected to achieve and demonstrate if he/she takes full advantage of the 
learning opportunities that are provided. It indicates the contribution that it makes to 
the programme(s) and specifies the learning facilities. 



1. Course/module 
title and (if 
applicable) 
reference number 
or code 




2. Programme title(s) 
and degree title(s) 
to which the 
course/module 
contributes 




3. Credit value 




4. Academic member 
of staff responsible 




5. Primary language 
of instruction and 
assessment 




6. Teaching location 
(if different from 
the main campus) 




7. School/Faculty and 
Department/Centre 




8. Modes of 

Attendance and 
Learning 




9. Any other external 
contributions to 
learning 




10. Arrangements for 
routine review and 
report 
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11. Date of 

production/revisio 
n of this template 



12. Aims of the Course/Module 



13. Learning resources 
13.1 Accommodation 



13.2 Course teaching team 



13.3 Additional visiting or occasional professors/speakers 



13.4 Library services 



13.5 Essential core texts 



13.6 Recommended additional texts, journals, wider reading 



13.7 IT and computing facilities 



13.8 Dedicated e-mail address/website 



13.9 Other specialist equipment 



14. Intended Learning Outcomes (ILOs), and Teaching, Learning and 

Assessment Methods 

A. ILOs for Knowledge and Understanding 

e.g. 
A1. 
A2. 
A3. 

A4. 

Teaching and Learning Methods 



Assessment methods 



B. ILOs for subject-specific skills 

e.g. 
B1. 
B2. 
B3. 
B4. 
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Teaching and Learning Methods 



Assessment methods 



C. ILOs for thinking skills 

e.g. 
C1. 
C2. 
C3. 

C4. 

Teaching and Learning Methods 



Assessment methods 



D. ILOs in other skills relevant to employability and personal development 

e.g. 
D1. 
D2. 
D3. 

D4. 

Teaching and Learning Methods 



Assessment methods 
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Annex C: The Self-Evaluation 

1. This annex offers guidance on the conduct of self-evaluation and the preparation of the 
self-evaluation report (SER), including the presentation of statistical data on student 
progression and achievement. 

The process of self-evaluation 

2. Self-evaluation is central to quality assurance and to the subject review method. The 
institution is invited to undertake this as a start to the review process. It will lead to a self- 
evaluation report (SER). It should be an analytical, evaluative, evidence-based account 
providing the context in which the programme is offered, the distinctive features and an 
assessment of the strengths, weaknesses, opportunities and threats to the effective delivery 
of the programme. Although it will be strategic in its evaluation, it is also expected to focus on 
recent, current and prospective students' interests. 

3. The process of self-evaluation brings a number of benefits. These include: 

The clarification of the aims, an improved specification of the programme(s) under review, 
The sharing of insights by staff and others with an interest in the quality and standards of the 
programme(s) into its nature and distinctive features 
The organisation of evidence such as performance indicators. 

4. Those with a legitimate interest in the educational programmes (the key stakeholders) 
include the academic staff, visiting professors and others engaged in the learning and 
assessment such as supervisors of internships and external evaluators, other staff in the 
institution who make a contribution to the programme (such as librarians), the students, 
alumni and representatives of the employing community. A self-evaluation process should 
ideally capture the structured comments of all these interests. 

5. The process requires considerable planning and organisation in the institution and should 
lead not only to a self-evaluation report that will form the basis of an external review but also 
to the sustainable development of internal quality assurance processes. The facilitator can 
play an important role in supporting this process. Institutions may also wish to consider 
nominating from the outset a "critical friend" as a reader of the draft self-evaluation report to 
assist the department or faculty /school in producing a clear, comprehensive and concise 
report that is fit for purpose. 

The Self-Evaluation Report (SER) 

6. The self-evaluation is the subject provider's evaluation of the quality of the student learning 
experience and the level of the students' achievements expressed in terms of the academic 
standards attained. Performance is measured against its own aims and the intended learning 
outcomes (ILOs) as determined by the institution using appropriate external reference points. 
Consequently, the SER needs to include a clear statement of the programme'(s) aims and 
ILOs (contained in a programme specification for each programme being evaluated - please 
refer to Annex B of this Handbook) together with an evaluation of the programme(s) of study 
designed to meet them. The evaluation is organised within the structure given by the aspects 
of provision, and should be supported by evidence. 

7. Generally, the SER will be written for the subject review arranged with the HFE and should 
address the programme(s) being presented for external review. 

8. The SER should discuss both strengths and weaknesses in the provision. Where 
weaknesses are acknowledged, the self-evaluation should include discussion of the actions 
planned and taken to improve academic standards and the quality of the programme. 
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9. As a rough indication of length, the SER, excluding the programme specification(s) and 
annexes, should not exceed 5,000 words and the Framework section (see below) should not 
exceed 800 words. It should be presented as a word document in electronic format in not 
less than a 10-point font and sufficient hard copies in print supplied to the visiting reviewers, 
the interpreter and the facilitator in the base room. 

Timetable for the submission of the self-evaluation 

10. The stages given in weeks before the site-visit are approximate and based on the 
minimum desirable. Where feasible, the HFE will work in partnership with the institutions 
to provide as much lead-time as possible. 

1 1 . The normal timetable for self-evaluation leading to an external review is as follows: 



Stage 1: Consultations in the higher education community on the scope of a proposed 
subject review programme: 40 weeks 

Stage 2: Workshop for representatives of institutions participating in a subject review 
programme: 30 weeks 

Stage 3: HFE confirms submission date for SER with the institution: 28 weeks 
Stage 4: Institution prepares an action plan for the self-evaluation process: 28 weeks 
Stage 5: Institution begins self-evaluation and in parallel prepares or refines its relevant 
programme specif ication(s): 26 weeks 

Stage 6: HFE confirms date of site-visit and details of the visiting review team: 1 0 weeks 
Stage 7: Institution submits its self-evaluation report to the HFE: 8 weeks 
Stage 8: The review chair appointed by the HFE to lead the external review contacts the 
institution to arrange details of the site-visit and offers an analysis of the SER together 
with any necessary points for clarification: 4 weeks. 

Stage 9: The reviewers prepare their initial commentaries on the SER and circulate them 
in the review team and with the facilitator: 1 week 

Stage 10: Site visit commences. The institution may wish to offer any essential update on 
developments since the SER was submitted that have direct and substantial relevance for 
the review. 



Use of the Self-Evaluation 

12. The SER informs and shapes the activities of the review team during the subject review 
visit. Reviewers test the rigour of the self-evaluation and the extent to which it presents an 
accurate picture of the academic standards and the quality of education that students on the 
programme(s) receive. Reviewers report on the rigour and openness of the self-evaluation, 
and the steps taken to improve the academic standards and quality of programme(s), under 
the Effectiveness of Quality Management and Enhancement aspect. 

13. An analysis of the self-evaluation determines the balance of the reviewers' activities. It 
informs and shapes the activities of the review team, but does not limit the scope of their 
enquiry. Reviewers may develop independent lines of enquiry in the light of the evidence 
gathered during the visit. 

Structure of the Self-Evaluation Report (SER) 

14. The SER should include three main sections: 

a. The Framework containing the programme specification(s). 

b. The Evaluation of the academic standards and the quality of the learning opportunities. 

c. The Annexes. 
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a. The Framework 

15. The Framework provides the context for the educational programmes that are being 
evaluated, a brief summary of the scale of the provision and the relevant programme 
specification(s). It will form the basis of the Introduction to the subject review report and be 
essentially factual and contextual. The main contents of the framework are: 

1) A title page should clearly indicate the subject review (e.g. Law), 
institution, faculty/school and department in which the 
programme(s) under review are located, and the date. 

2) The Mission Statement of the Institution 

3) A summary of the main educational aims of the subject being 
reviewed 

4) A full list of the academic programmes provided in the subject, 
together with a clear indication of the programme(s) which are 
included in the self-evaluation and those which are to be in 
scope of the external subject review. 

5) An indication of the scale of the provision, in the context of the 
University using the following data for the most recent academic 
year on the student and staff profile should be included: 

• number of students in the institution 

• number of students (all years) on each programme in the 
subject, the number enrolled in the most recent year 1 and 
the number of graduates in the most recent graduation 

• number of HE academic staff employed by the institution, 
by grade 

• number of HE academic staff contributing to the 
programme(s) under review by grade 

• number of non-academic support staff supporting the 
programmes under review, including technical, 
administrative, clerical and IT staff. 

1 6. In addition, the institution may wish to provide a factual explanation of the context in which 
the programme(s) are provided. This will involve consideration of the Mission of the University 
and how the work of the faculty/school contributes to the achievement of that mission through 
the programme(s) which it provides. There might also be some consideration of how the aims 
and ILOs of the programme(s) support this contribution. It is useful in this context for the 
institution to give the precise location of the programme(s) under review (e.g. the 
department). 

17. The programme specification(s) should then be either inserted in the text or cross- 
referenced to another word document that should accompany the SER. The programme 
specification(s) should be presented using the template in Annex B1 or an equivalent 
structure. 

b. The Evaluation of the Quality of Education 

18. This section should contain the subject provider's evaluation, with supporting evidence, of 
the academic standards achieved by students compared to the stated aims and ILOs that are 
confirmed by the institution using appropriate external reference points, and the quality of the 
education provided for students, measured against these stated aims and ILOs. The 
evaluation should be set out in the structure provided by the six aspects of provision, namely: 

Curriculum Design, Content and Organisation 
Teaching, Learning and Assessment 
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Student Progression and Achievement 
Student Support and Guidance 
Learning Resources 

The Effectiveness of Quality Management and Enhancement. 

19. Information on the key features and relationships for each aspect of provision is provided 
in the Subject Review Aide Memoire (Annex A). 

c. The Annexes 

20. The SER may include annexes to provide more detail as well as statistical information. 

21. Details of students' progression and achievement are required and should include the 
following data for the three most recent cohorts for each programme. 

• analysis of entry qualifications 

• number and percentage of students passed and failed at the end of each 
year/semester 

• number and percentage of withdrawals 

• number and percentage of transfers in and out of the programme(s) 

• number and percentage of entrants completing the programme(s) 

• number and percentage of qualifications awarded (for example, certificate, diploma 
and degree results, including classifications where relevant) 

• number and percentage of graduate destinations (employment and further study). 

22. For the effectiveness of the programme(s) in contributing to the Mission of the University 
and faculty/school to be reviewed, there should be an analysis of the destinations of 
graduates. This should include an analysis of the statistics of students entering 'appropriate' 
employment or further education either within the subject area studied, or outside the subject 
area, but in 'appropriate graduate level' employment. 

23. Other annexes may provide more detailed information on the steps taken to engage with 
key stakeholder groups such as the arrangements to survey and analyse levels of 
satisfaction. 
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Annex D: Peer Review 
Introduction 

1 . Peer review is an essential part of most quality assurance systems. It forms the basis of the 
published method used by Al Hussein Fund for Excellence (HFE) in Jordan. This annex 
summarises the characteristics of peer review and sets out the criteria for the appointment of 
reviewers, the personal specification for reviewers and review chairs and criteria for the 
composition of review teams. 

2. Review by peers means that the institution has a reasonable expectation that the visiting 
reviewers should hold, or have recently held, equivalent professional positions as those with 
whom they conduct their enquiries in the institution. They should have the confidence of the 
institution and, when offering professional opinion on their area of expertise, they are credible 
in the eyes of the institution. Peer review also offers safety in numbers for the process, in that 
the team as a whole provides a degree of protection for the institution and the HFE from one 
person's eccentric views. 

3. The effective contribution of peer reviewers is underpinned by the application of criteria for 
their appointment, equal opportunities for all reviewers, experience of the same training and 
support in the quality assurance and accreditation process. In addition, the specification for 
the composition of review teams offers transparency to the process of arranging reviews. The 
HFE will make appropriate arrangements to monitor the reviews to satisfy itself and the 
institution that the peer review process respects the published method and protocols. 

Recruitment, Training and Visit Allocation 

4. Reviewers are introduced to the HFE by the QAA in the UK which recruits and trains all 
reviewers in line with standard operating procedures and published criteria to ensure that they 
make an effective contribution to the process. The HFE makes appointments of reviewers 
based on the assessment of the needs of the review programme and the recommendations of 
the advisor on behalf of the QAA. The HFE also provides specific briefing for reviewers. The 
criteria for appointment are: 

• All reviewers actively engaged in the schedule of engagements meet the specification 
(see below) 

• All reviewers allocated to a review have successfully completed the training 

• Reviewers are provided with the Subject Review Handbook together with supporting 
materials and guidelines 

• Reviewers will be allocated to reviews that are within their competence 

• Reviewers will make themselves available for the whole of the scheduled review 

• Reviewers will not be assigned to a review where either they or the institution believe 
there to be a potential conflict of interest (see below, Composition of a Team)) 

• Reviewers take a professional interest in the process and the advancement of 
academic affairs. 

Personal Specification 

5. Reviewers need to have sufficient status and reputation for their views to be respected in 
the academic community. They also need to bring to the process a high order of skills in 
communication and evaluation. All candidates for the role of reviewer will be invited to submit 
a CV. The HFE will give the reviewer a contract in the form of terms of reference. 

6. The following points represent a core specification: 
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Essential 

• Academic expertise in one or more discipline that appears in the schedule for review 

• Current or recent academic experience including successful teaching practice and at 
least five years teaching and/or research within the last ten years, or 

• Those in professional practice in a relevant discipline who have recent, direct 
experience of academic activity may also be considered 

• Current or recent experience in quality management and improvement projects or 
systems which have made an impact 

• Proven abilities in communications in English including: listening; joining group 
discussion; respecting the views of others; leading (chairing) group discussion; rapid 
reading with understanding; and concise clear writing to tight deadlines 

• Competence in the use of and interpretation of number including: the accurate 
analysis of data sets; verification and reconciliation techniques; presentation of valid data 
in support of a judgement 

• Proven ability in evaluation including: appraisal of the context; identifying logical or 
irrational argument; making sound judgements based on facts; adjusting judgements in 
the light of additional information or well-argued alternative views in a professional 
context; and a willingness to justify judgements. 

Desirable 

• A favourable disposition to initiatives to improve the quality and academic standards 
of higher education 

• IT skills, including the use of laptops or notebooks, internet and intranet. Preferably in 
MS word 

• Current or recent experience in moderation of marking, external examining and/or 
formal validation of graduate attainment 

• Effective practice in curricular developments, including the writing of outcome-related 
curricula documents, action plans for programme/course improvements or strategies for 
learning, teaching and assessment 

• Acknowledged track record in research and other scholarly activities 

• Recognised contributions to society or the community within the normal range of 
academic activities (e.g. projects, consultancy, teaching, coaching or mentoring) 

• Advisory or interventionist functions as internal or external consultant or change 
agent in higher education or related professional fields. 

Review Chairs 

7. Review chairs will meet all the above requirements, except that relevant academic activity 
in the discipline under review is not always required, and in addition will need to demonstrate: 

• Recent experience in internal and/or external review methods 

• Proven qualities of leadership and the management of people and information in task 
groups or projects 

• Abilities to implement procedures and protocols consistently yet fairly to 
accommodate local circumstances 

• Effective chairing of reviews and meetings including thorough planning, collaboration 
with other key participants and time management 

• Ability to assess the evidence available and the validity of emerging judgements 

• Ability to write cogently to deadlines and edit the writing of reviewers to meet the 
specification for the review report 

• Ability to evaluate the review and make constructive suggestions for the continuing 
improvement of the method 

• On request, additional contributions to the process through, for example, 
conferences, editing the review reports generated by others, trawling reports in order to 
draft overview or summary reports, and the preparation of materials for briefing reviewers 
and institutions. 
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Composition of a review team 

8. The HFE will compose each review team for each review, taking advice from its appointed 
consultant in the subject review programme. The HFE will consult with the institution and will 
inform it of the proposed composition prior to its confirmation. However, the final allocation of 
reviewers is made by the HFE to ensure the independence of the review process. 

9. The key criteria for the composition of the team are as follows: 

• Teams are composed of reviewers who meet the above requirements 

• The minimum number of reviewers will be two plus a review chair. The size of the 
team will vary according to the scale and complexity of the academic programme(s) 

• The team will be led by a review chair who may or may not have relevant expertise in 
the discipline 

• The profile of the team reflects the profile of the main academic activities of the 
programme(s) being reviewed 

• The team cannot cover every specialist teaching and research interest, but the HFE, 
guided by the institution, will seek to provide a balance of interests in the principal 
academic activities 

• Where appropriate, a team may include a reviewer from professional practice 

• Potential conflicts of interest in the team will be avoided, and the HFE will seek the 
cooperation of reviewers and the institution to this end. 
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Annex E: Preparations for the site visit including documentation to be 
provided by the Institution 

Making Introductions 

1. Al Hussein Fund for Excellence (HFE) will arrange with the institution to establish the 
names and roles of key representatives of the institution and the faculty/school and 
department directly involved in the subject review. It will provide the institution with the names 
and e-mail addresses of the members of the subject review team. It will introduce the principal 
parties to each other by e-mail. The review chair will normally contact the named 
representative in the faculty/school who is leading the arrangements for the subject review not 
later than four weeks before the site-visit. 

Preliminary discussions by e-mail 

2. The review chair will: 

• introduce him/herself to the named representative and to the institutional facilitator 

• offer feedback on the self-evaluation report, based on an analysis, which may include 
suggesting that the institution supplies additional information 

• request a website reference to access essential contextual information equivalent to a 
printed prospectus 

• arrange with the representative a suitable outline schedule of activities for the site 
visit based on annex F and fine-tuned to local circumstances 

• confirm logistical arrangements for the site visit including daily travel, the base room, 
the range of supporting documentation to be available, and other details such as access 
to a web terminal, printer and photocopier 

• agree the sample of student work to be available in the base room 

• offer advice as requested on matters such as the review method, the role of the 
facilitator and who should be invited to meetings. 

Documentation to be provided by the Institution 

3. The following items in English should be sent to the review team six weeks 
in advance of the visit and printed duplicate copies of this material should also 
be provided in the base room for each reviewer and the interpreter/translator: 

• a copy of the self-evaluation report (SER) including the programme specification(s) 

• a supplement if necessary containing updated information; for example, the student 
intake and progression data for the most recent cohort 

• the main subject information provided for students (for example, subject/course 
handbook) or the website reference. 

4. In addition the following documents should be available in the base 
room: 

• internal reports on the programme 

• if they are in use, external examiners' reports and accreditation reports for the 
programme(s) for the three last years, with guidelines and protocols for external 
examiners where these are used 

• teaching/learning timetables for each programme for the period of the visit 

• HE academic staffing list and brief CV profile (giving main teaching/research interests 
and administrative responsibilities) 

• internal subject reports for the last three years 

• The summary reports of the outcomes of recent student satisfaction surveys and any 
similar surveys of alumni and employers 

• minutes of relevant meetings 



58 of 79 



Al Hussein Fund for Excellence: Subject Review Handbook January 2005 



• module or course specifications or equivalent outlines and relevant associated 
documentation (for example, booklists), typically for the core modules and a range of 
specialist options for the programme(s) 

• any supporting, additional evidence relating to each aspect, to exemplify and support 
statements made in the self-evaluation. 

Student Work 

5. This should typically include a sample of the work of the most recently assessed cohort for: 

• each level and year of study 

• the core modules, units or courses and a range of specialist options for each 
programme of study 

• a representative range of attainment/marks 

• a range of assessment methods, for example continuous assessments/coursework; 
practical/laboratory work and projects; videotapes and artefacts; and examination scripts, 
essays and dissertations. 

6. Question papers, marking and feedback sheets, and assessment criteria (where they are in 
use) should accompany the samples. 
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Annex F: Typical outline schedule for site-visit of 2.5 days 

1. This outline schedule is offered as a basic model suitable for adapting to local 
circumstances. The review chair will agree an outline with the departmental representative 
prior to the site-visit. Times may change, and all such timetables need to be flexible. If the 
President of the Institution wishes, the team will be happy to meet him or his representative at 
any convenient time. It is sometimes possible to arrange the introductory meeting late in the 
afternoon or the day before day 1 (day 0). The institution may also wish to pre-arrange some 
key meetings that will take time to set up such as the meeting with past students and 
employers. The priorities for lines of enquiry and the best use of time will be discussed at the 
first review team meeting. The two versions are arranged to serve a site-visit that starts at 
0830 on day 1 , and a site visit that is scheduled to start midday. 

2. The team may wish to observe a small number of classes (about 3). It would welcome 
suggestions with regard to suitable classes. A final decision regarding class observations will 
be addressed at the first meeting to finalise the agenda. 

3. With the exception of meetings with students (past and present) and employers, and the 
private meeting of the team at the end of the third day, the IF is invited to every meeting. 



Day1 0830 


Meeting between reviewers, Departmental 
Representative and Institutional Facilitator (IF) 
to finalise the agenda for the review; 
discussion of any practical questions; 


1 /2 hour 


0900 - 1 100 


Meeting on Academic Standards with all 
faculty. This will include aims, ILOs, use of 
external reference points, curriculum, 
assessment, student progression and 
achievement. 


2 hours 


1115-1215 


Meeting with current undergraduate students 
(20-30) 


1 hour 


1215-1300 


Tour of Learning Resources 


% hour 


1300 


Light working lunch for team in base room 




1300-1700 


The team will want to read the documentation 
in the base room and may wish to meet some 
faculty or other staff for discussion. They may 
wish to observe some classes 


4 hours 


1700-1800 


Team Meeting, with IF present 


1 hour 


Day 2 08.00 


Meeting between reviewers, Dept Rep and IF 
to finalise the days activities 


Vi hour 


0900 - 1100 


Meeting with faculty on Quality of Learning 
Opportunities. This will include teaching, 
learning, student support and guidance, and 
all aspects of resources 


2 hours 


1115 - 1215 


Meeting with staff on Effectiveness of Quality 
Management and Enhancement 


1 hour 


1300 - 1400 


Light lunch for team with past students 
(alumni) and employers (if possible) 


1 hour 


1400 - 1700 


The team will want to read the documentation 
in the base room and may wish to meet some 
faculty or other staff for discussion. They may 
wish to observe some classes. 


3 hours 


1700 - 18.00 


Team meeting with IF present, including 
identification of any significant gaps in the 
evidence base and agree agenda for the next 


2 hours 
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meeting 




Day 3 

0830-1030 


Clarification meeting with reviewers and all 
Faculty followed by team writing the key points 
emerging. 


Up to 2 hours 
(1 hour may be 
sufficient for the 
meeting) 


1030-1100 


Final collation of evidence and any further 
clarification in ad hoc meeting 


Vi hour 


1100-1200 


Private meeting of the team, no University 
staff will be present to finalise conclusions and 
feedback 


1 hour 


1200-1230 


Oral feedback of major strengths and 
weaknesses to Computing staff/University. 


Vi hour 




Day 1 

1430 


Meeting between reviewers, Departmental 

RoncocontQti\/o onH Inctiti itional Fooilitcitnip to 
RopicbcllLcUIVo dllU II loLUULIUI Idl rduM ILdLUI LU 

finalise the agenda for the review; discussion 
of any practical questions 


15 -30 minutes 


I UUU I DOU 


IVyiooti nn nn AraHornip QtQnrlarHc \A/ith qII 
IVIccLliiy UN ModUcllllL/ OLdllUdlUo W I LI I all 

faculty. This will include aims, use of reference 
points, ILOs, curriculum, assessment, 
progression and achievement 


i R hni ice 
I .O 1 IUUi b 


I DOU I / OU 


IV/looti n/-i \A/ith fi ircont i inHornrarli ioto cti irlontc 

ivi t?tun ly wilii uur r t;i i l ui luciyi duuctic oiuut?i lib 
(20-30) 


1 hni i r 
I IIUUI 


i / ou i o i j 


Xniir nf 1 pprninn RpQniirppc: 
i uui ui i— ccii i in ly ncouui vco 


3/a hni i r 

/4 IIUUI 


1815-1915 


Team meeting with IF present 


1 hour 


Day 2 08.30 


Meeting between reviewers, Dept Rep and IF 

tr\ finolico tho Hq\/c Qftiwitioc 
LU lllldlloc LI Ic Udyo dOLIVILICO 


1 /2 hour 


0900 - 1100 


Meeting with faculty on Quality of Learning 
Opportunities. This will include teaching, 
learning, student support and guidance, and 

qII Qcnoptc nf rocni irpoc 
dll dofjyi/lo UI icoUUIOco 


2 hours 


1115 - 1200 


Meeting with CS staff on Quality Management 
and Enhancement 


1 hour 


1300 - 1400 


Light lunch for team with past students and 
employers (if possible) 


1 hour 


1215-1300 
1400-1700 


The team will want to read the documentation 
in the base room and may wish to meet some 
faculty or other staff for discussion. We may 
wish to observe some classes 


4 hours 


17.00-1800 


Team Meeting, with IF present 


1 hour 


Day 3 08.30 


Meeting between reviewers, Dept Rep and IF 
to finalise the days activities 


Vi hour 


0900-0930 


Further reading and writing and possible ad 
hoc meeting in response to emerging lines of 
enquiry 


Vi hour 


1000-1300 


The team will want to read the documentation 
in the base room and may wish to meet some 
faculty or other staff for discussion. We may 
wish to observe some classes 


3 hours 


1300-1400 


Team meeting with IF present to review 
evidence base, identify any significant gaps 
and agree agenda for next meeting 


1 hour 


1400-1600 


Clarification meeting with all Faculty followed 
by team writing key emerging points 


Up to 2 hours 
(1 hour may be 
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sufficient for the 
meeting) 


1600-1700 


Private meeting of the team, no University 
staff will be present, to finalise conclusions 
and key points for feedback 


1 hour 


1700-1730 


Oral feedback of major strengths and 
weaknesses to staff/University. 


1 /2 hour 
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Annex G: Protocol and Prompts for Meetings between Reviewers and 
Students during Reviews 

Purpose and conduct of meetings 

1. The views of students represent an important part of the evidence collected by the review 
process. The reviewers will be seeking to focus on key points emerging from their reading of 
the self-evaluation report and supporting information. They will wish to reflect on the 
expressed views following the meeting and consider the full range of information when 
making their judgements. 

2. The list of prompts below should be used to plan the meeting beforehand and the lines of 
enquiry should be selective, based on the self-evaluation report and information emerging 
earlier in the review site-visit. The meeting should not normally take longer than 60 minutes. 
One of the review team will take notes of the areas addressed and the responses. In the 
notes, comments will be generalised, recognising alternative views, and not ascribed to 
individuals. 

3. The meeting should be pre-arranged to include a representative group of students across 
all levels with a range of modules within the educational programme. When more than one 
programme is being reviewed, the review chair will arrange with the departmental 
representative and the facilitator an appropriate representation either within one meeting or in 
separate meetings. The meeting may include members of students' councils, but should not 
be "packed" with politically active students. The review chair normally chairs the meeting. The 
ideal numbers are approximately 20. The institutional facilitator and members of staff of the 
faculty should not attend. 

4. Where practicable, the language should be English but, with the general agreement of 
those present, detailed points may be better expressed in Arabic and these should be briefly 
translated for the benefit of all reviewers. Dialogue should be constructive and should avoid 
personalising criticism of staff. Questions from reviewers should be open and unbiased. One 
apparently extreme opinion may be checked out around the whole group for a consensus or 
dissention. The review team may decide to break a large group into smaller groups after the 
Introduction, with one reviewer leading each. 

Structure of meeting 
Introduction 

The review chair will introduce the visiting reviewers and provide a brief summary of the 
purposes of the visit and the review method. Ideally they should have already seen a copy of 
this agenda. Students should be invited to introduce themselves by name, programme, 
year/level and reasons for choosing programme/institution. 

Have any students present been involved in the institution's annual review and reporting 
process recently, or the preparation of the self-evaluation report (possibly known to them as 
self-study) (see also final section below)? 

Curriculum and standards (intended outcomes, curriculum, assessment progression 
and achievement) 

Reasons for choosing the programme 

Clarity of stated aims, intended learning outcomes and what is assessed when. 

Match between curriculum and expectations (for example, flexibility, choice and content). 

Relevance of the curriculum (knowledge and skills) to prospective career/further study. 

Any examples of exposure to current or recent research and professional practice via staff in 

the institution. 

Opportunities for practical, vocational experience, internship and projects where appropriate. 
Student timetable and workload. 

Sense of challenge and/or achievement of intended learning outcomes. 
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Perceptions of added-value (gain in knowledge and skills, with particular reference to 
sustainable or independent learning and new insights). 

Teaching and Learning 

Range of teaching and learning methods experienced 
Students' views on quality of teaching 
Course materials, including learning packs 
Guidance and support for independent study 

Support and guidance 

Admission and induction procedures 

Quality of written information and guidance on the programme 
Access to tutors and arrangements for academic and personal advice 
Support during periods of practice, study abroad, internships and other 
off-site experience 

Facilities for disabled and distinguished students 
Feedback on assessed work 

Quality of information given to students on their progress and any areas that may require 
additional study 

Match of careers advice and guidance to career aspirations. 
Learning Resources 

For all this section, how well are they USED? (For example, is there an insistence that 
learning tasks and assessed assignments have to demonstrate their use?) 
Library services (opening hours, practical access, and user support, availability of stock - 
standard texts, journals and wider reading) and do you USE them? 

IT provision (opening hours, practical access, searching, user support, availability of 
terminals, internet and intranet, e-mailing) and do you USE them? 
Specialist equipment, including relevant software - and do you USE it? 
Teaching halls, including laboratory or studio provision 
Recreational facilities - and do you USE them? 

Space for study, or other independent learning, including practical projects, common rooms, 
refectory, social areas - and do you USE them? 

Effectiveness of Quality Management and Enhancement 

Ways in which students' views are sought and frequency 
Representation on council or committees 

The degree to which students' views is influential, with examples of action, impact and 
feedback 

Students' contribution to the self-evaluation report 

In brief, in the students' view, what works (strengths), what should be improved, and are they 
confident that the quality management process captures these features? 

Conclusion 

Students should be given the opportunity to raise points not covered by the Reviewers' 
questions 

The review chair will thank the students for their contribution. 
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Annex H: Protocol for Class Observation and class observation form 
Purposes of class observation 

1. Teaching and learning in classes lie at the heart of educational programmes. Peer review 
can make a significant contribution to the enhancement of the quality of education when 
applied effectively to the evaluation of teaching and learning. As part of the review process, 
both for internal quality assurance and for the external reviews arranged by the HFE, 
reviewers may wish to observe a sample of classes and evaluate the contribution they make 
to the attainment of the stated intended learning outcomes (ILO). The integrity of teaching and 
learning in classes should be highly valued and reviewers need to be sensitive to the possible 
impact of intervention. Any observation of classes must respect this integrity and follow the 
protocol set out in this annex. 

2. The purpose of observing classes is to collect evidence by direct observation of the quality 
of the teaching and learning and to draw reasonable inferences on, for example, the 
appropriateness of the teaching methods and the classroom facilities. The purpose is not to 
appraise the performance of academic staff. Any evaluations made will not be personalised 
nor over-generalised. When inferences are drawn from a sample of classes, these should be 
checked against other sources of evidence such as annual reports and discussions with 
academic staff and students. 



Peer review in internal systems 

3. As part of the development of internal systems for quality assurance, institutions may wish 
to develop suitable processes and protocols for peer review of teaching and learning. This 
protocol may assist them. Any internal peer review of teaching and learning should have a 
clear purpose that distinguishes between, on the one hand staff appraisal systems (where 
information on staff performance is gathered and used to inform management decisions on 
performance, rewards and needs for further staff development) and, on the other hand peer 
review processes that give insights into current practice, effectiveness and potential for 
enhancement. The outcomes of internal peer review may be used as part of the evidence 
base in quality assurance systems but, when presented or summarised for self-evaluation 
reports or for external review, the information should not identify individual staff or students by 
name. 



Peer review in external reviews 

4. For external reviews arranged by the HFE the visiting peer reviewers will determine the 
need for, and the number of classes in the sample for, class observation after considering 
their preliminary reading of the documentation and their initial written commentaries. The time 
available for observations is limited and the visiting reviewers have many calls upon their time 
during a site-visit. If the institution is able to present summative evidence of the outcomes of 
class observations undertaken as part of earlier internal review, this evidence can be helpful 
to the visiting reviewers. They may be able to reduce the size of their sample of observations 
to a degree that allows confirmation or verification of the outcomes of the internal review. The 
institution will need to make available to reviewers a schedule of classes. It can expect to be 
consulted at an early stage of a site visit on the sample of classes that is of interest to the 
reviewers and on the practical considerations such as the language of instruction, 
accessibility or any special sensitivity. 
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Sampling 

5. The range of classes available should be considered together with the programme and 
course specifications when identifying a sample for observation. Large and small group 
teaching, practical sessions and workshops, directed individual learning, the application of 
skills and distance learning may be included. Internships will not normally be included in the 
sample. 

Evaluation 

6. Reviewers will evaluate: the clarity of the classes' stated objectives and their relationship 
with the intended learning outcomes stated in the course and programme specifications; the 
extent to which the class contributes to the specifications; the appropriateness of the selected 
teaching methods and the levels of participation by the students; the effectiveness of the 
teaching of subject knowledge, including references to wider reading and to recent and 
current research or professional activity; the effectiveness of the development of relevant 
subject-specific, transferable and practical skills; and the appropriateness of the facilities 
(accommodation, equipment, use of texts and other teaching aids). It is not expected that all 
of these will be on display in every class. 



Good practice 

7. Before observing a class, the peer reviewer should read any existing background 
information, such as the course and programme specifications and the student or course 
handbook, and meet the lecturer briefly for up to about 10 minutes. The purpose of this 
preparation is to establish the context for the class, its place in the sequence of the teaching 
programme, its objectives, and the lecturer's intentions for the chosen methods of teaching 
and learning. At this meeting, the reviewer should also confirm that the lecturer understands 
the protocol and in particular the purpose of the observation. 

8. The peer reviewer will not normally attend a class for less than 45 minutes or for more than 
one hour. In the case of a long session of up to three hours, the reviewer may arrange with 
the lecturer to attend in short spells at the beginning, middle and end. The reviewer is strictly 
an observer and in no circumstances should intervene or take part in any way in the teaching 
or group activity. 

9. The reviewer may arrange with the lecturer to hold a brief discussion (not more than a few 
minutes) shortly afterwards in a private place to share reflections on the effectiveness of the 
class. The discussion can clarify any points that arise and will provide an opportunity for the 
member of staff conducting the class to offer a self-evaluation of the effectiveness of the 
session if they wish. 

10. The reviewer should complete a record for the class observation shortly afterwards, using 
the standard observation note below. 



The use made of the evaluation 

11. The evaluation made by the reviewer will form part of the evidence base for the review, in 
due proportion to the size of the sample and the time devoted to class observations as a 
fraction of the whole review schedule. The evaluation will be considered within the review 
team, but will not be divulged to the institution. The institution should not seek to use the 
evidence base from class observations as a means to appraise staff performance. 
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Observation note 

(For use in all teaching and learning sessions - including lectures, tutorials, practicals and 
students' independent learning sessions) 

Institution 
Programme 
Subject /course 
Reviewer 

Date/session/time 

Length of observation (minutes) 

Level/year/mode, e.g. FT/PT 

Number of Students present 

Type of activity, e.g. lecture, practical 



Are the learning objectives planned for this session clear and are they appropriate to 
the ILOs for the course and the programme? (e.g. knowledge and understanding, key 
skills, cognitive skills, and subject-specific, including practical/professional skills)? 



Are the teaching methods and range of activities appropriate? 



Is the content appropriate for the ILOs and to the level of study? (Knowledge and skills 
development in terms of currency, accuracy, relevance, use of examples, level, match 
to student needs) 



Do the opportunities for the students to participate match the programme 
specification's expressed aims to develop independent learning? 



Are the facilities appropriate and are they used effectively? (Accommodation, 
equipment, use of websites, texts and other teaching aids) 



Please summarise the session's overall effectiveness in contributing to (i) the learning 
objectives for the class and (ii) the programme and course specifications: 
(e.g. indicate:- appropriate in meeting the ILOs; ineffective; matters that should be 
enhanced; example of good practice) 



Please indicate relevance to other aspects: 



CDCO 


TLA 


SPA 


SSG 




EQME 










LR 
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Annex I: Student Work Review Note 



Institution 

Programme 

Reviewer 

Date 

Subject 

Module/Course title 

Level/Year of study 

Number of items of work in the sample 

Relation of the sample to the spread of student marks for the course 
Nature of student work: 

Essay 

Examination Script 

Lab or Workshop Project 

Other (please specify) 



1. Academic Standards: does the assessment match intended learning outcomes for the 
module/programme; are the standards set appropriate to the level of study; do they reflect 
any accreditation or other external reference points such as professional requirements? 



Grade 

2. Student Achievement: are the intended learning outcomes appropriately achieved at the 
level indicated by the grade/class awarded? 



Grade 

3. Quality Assurance and Staff Input: are the marking criteria clear and consistently 
applied; is there evidence of moderation of marking; is the quality of the feedback 
appropriate? 



Grade 

4. Overall Comments: identify in particular, and indicate in boxes below, any evidence that 
relates to academic standards (CDCO and SPA), quality of learning opportunities (TLA, SSG 
and LR) and the effectiveness of quality management and enhancement (EQME). 



Overall Grade 



Academic Standards 


TLA 


SSG 


LR 


EQME 


CDCO/SPA 
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Annex J: Meeting Note 

Institution 

Programme(s) 

Meeting/aspect 

Persons attending 

Reviewer(s) attending 

Date 

Time start/finish 

Recorder 




Conclusions 



Follow up points 

1. Used references to source material e.g. SER 

2. Cross reference to other meetings/sources of evidence 

3. Matters for further exploration 

Circulation to: 
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Annex K: Review Team Meetings Agenda 
Review Team Meeting 
Day One - Afternoon 

Typical Agenda 

1 . Introductions, including clarification of agreed roles and the respective responsibilities 
of review chair, subject specialist reviewers, institutional facilitator and interpreter/translator. 

2. Reminder of review method: 

• evaluate the degree of success in attaining the subject provider's aims and the ILOs 
as set out in the programme specification 

• evidence needed to support judgements 

• criteria for assigning grades. 

3. Consideration of subject specialist reviewers' division of responsibility. Reminders to: 

• share evidence with appropriate colleagues: agree the mechanisms 

• use Subject Review Handbook for guidance, including Subject Review Aide Memoire 
(Annex A) 

4. Identification of issues to be pursued: 

• key points arising from the reading of the self-evaluation report 

• discussion of brief initial commentaries prepared by reviewers 

• priorities for lines of enquiry 

• use of the outline schedule of activities and any fine-tuning necessary 

• discussion of process of developing written summaries. 

5. Strategy to be adopted to ensure appropriate coverage of student work, and 
consideration of need for a sample of class observations. Reminders and discussion of: 

• protocol for observation and feedback to teaching staff 

• use of Observation Note (Annex H) and Student Work and Assessment Note (Annex 
I). 

6. Meetings with staff, current and former students, and others. Reminders to: 

• use of Subject Review Aide Memoire (Annex A) and Aide Memoire for Meeting with 
Students (Annex G) 

• keep, use and circulate notes of meetings. 

7. Review programme for the following day: 

• agree coverage of meetings 

• agree coverage of class observations 

• agree review of student work and supporting documentation 

• time and location of team meeting(s). 

8. Logistics: 

• transport, security of base room, administrative support, drinks and meals, photocopying 
and use of telephone 

• procedures for requesting additional documentation and making appointments with 
staff. 

9. AOB. 
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Review Team Meeting 
Day Two - Early Evening 

Typical Agenda 

1 . Discussion of the day's activities. Review evidence gathered: 

• any class observations 

• meetings held and information gathered: summary of key points shared with team 

• coverage of programmes/courses/modes/levels 

• sampled student work and other documentation read 

• issues to be tracked further in relation to each aspect of provision. 

2. Consideration of the quality of the self-evaluation report. 

3. Review evidence gathered in relation to each aspect of provision. Share written 
summaries of evidence. Give initial consideration to the appropriate grading for each aspect, 
identifying where additional information and/or clarification is required. 

Curriculum Design, Content and Organisation 
Teaching, Learning and Assessment 
Student Progression and Achievement 
Student Support and Guidance 
Learning Resources 

Effectiveness of Quality Management and Enhancement 

4. Review programme for the following day: 

• agree coverage of meetings 

• agree review of student work and external examiners' reports 

• agree any further coverage of observations 

• time and location of team meeting(s). 

5. Arrangements for production of reviewers' written summaries of evidence and for the 
tabling of first drafts of evidence and judgements at the next team meeting. 

6. AOB. 
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Review Team Meeting 
Day Three 

Typical Agenda 

The institutional facilitator does not attend this meeting. 

1 . Summary of evidence gathered. Recap on the coverage of the previous evening and 
any new evidence. Debate on the quality of education provided in relation to subject aims the 
level of achievements in relation to the ILOs: 

• agree conclusions 

• finalise grading of each aspect of provision (subject to verification afterwards - not 
announced in the oral feedback). 

2. Arrangements for the oral feedback: 

• issues to be presented 

• conduct of meeting. 

3. Timescale for the production of the subject review report. Importance of returning 
comments on draft report. 

4. Arrangements for: 

• clearing the base room 

• returning documents and keys to institution 

• collation of documentation to be retained by the review chair 

• removal or shredding of confidential papers which do not need to be retained 

5. AOB. 
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Annex L: Grading the Aspects of Provision 

1. The graded profile shows the extent to which the institution has established and 
implemented its aims and ILOs, provides appropriate programme(s) leading to academic 
awards that are quality assured and meet the stated aims, all informed by the programme 
specification(s), the SER and the evidence base generated for and by the site-visit. 

2. The creation of the graded profile is achieved by applying a grade to each aspect of 
provision. There are four numerical grades on the scale: 1, 2, 3, and 4 in ascending order of 
merit. 

3. A grade of 2 or better in an aspect means that the students' learning opportunities 
and the level of student achievement in that aspect make at least an acceptable contribution 
to the attainment of the ILOs in the programme specification, and that the aims are being at 
least broadly met. 

4. A grade of 1 means either that the students' learning opportunities and the level of 
achievement in that aspect make an inadequate contribution to the attainment of the ILOs 
contained in the programme specification, or that the implementation of the ILOs in that 
aspect do not provide students with the learning opportunities and achievements that would 
support a judgement that the aims were being met. 

5. The criteria for assigning grades are set out below. 

6. The assignment of the grade for each aspect is a matter for the professional 
judgement of the reviewers, drawing on the evidence from both the self-assessment and the 
review visit. 

The Overall Judgement 

7. The overall summative judgement is derived from the profile. All the profile elements 
have equal weight. Any provision where one or more of the profile elements receives a grade 
of 1 is subject to the following procedures: 

• If, on first review, one or more of the profile elements receives a grade of 1, the 
quality of the education will be recorded as 'subject to further review within a year'. A 
review report containing this decision will be published. A second review will take place 
within 12 months, commencing with a management action plan, leading to a self- 
evaluation and a second site-visit. 

• If, after this further review, the profile still contains one or more elements graded 1, 
the quality of education will be recorded as 'unsatisfactory'. A second review report will be 
produced. 

8. A profile with all elements graded 2 or better will be reported as 'quality 
approved'. Where the graded profile includes three or more grade 2s, the HFE will 
invite the institution to write an action plan. The HFE may wish to arrange with the 
institution a brief peer review visit to explore the impact of the action plan at a later 
date. 

9. A grade of 3 or higher for CDCO, SPA and EQME indicates that the institution has 
demonstrated that this aspect is verifiable against comparable programmes in the Kingdom, 
regionally and worldwide for the academic standards represented in the degrees awarded by 
the institution. A grade 4 in any of these three aspects demonstrates a higher order of 
performance that (while still able to benefit from continuing improvement) is at least the equal 
of any similar programme, verified by the use of external referencing and evaluation. 
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10. The conclusions given in the oral feedback from the review team to the institution 
at the close of the review visit will identify strengths and any weaknesses, but grades 
will not be announced pending the verification of the review process and the draft 
subject review report arranged by the HFE in association with the QAA in the UK 
shortly after the completion of the subject review programme. 
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The Grade Descriptors 



Aspects of 
Provision 


Tests to be 
Applied 


To what extent do the students' learning opportunities and 
the level of student achievement, within this aspect of 
provision, contribute to meeting the ILOs contained in the 
programme specification? 


Do the ILOs, and the level of attainment of those ILOs, allow 
the aims set by the subject provider to be met? 


Scale Points 


1 


2 


3 


4 
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2. Teaching, Learning and 


provider are 


contribution 


contribution 


to the 


Assessment 




not met. 


to the 


to the 


attainment of 






This aspect 


attainment of 


attainment of 


the stated 


3. Student Progression and 


does not 


the stated 


the stated 


ILOs. 


Achievement 




support the 


ILOs, but 


ILOs; 








stated aims 


significant 


however, 


The aims set 


4. Student Support and Guidance 


and/or the 


improvement 


there is 


by the 






ILOs; there 


could be 


scope for 


subject 


5. Learning Resources 
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improvement 


provider are 






shortcoming 






met. 


6. The Effectiveness of Quality 


s that must 


The aims set 






Management and Enhancement 


be rectified. 


by the 


The aims set 
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by the 










provider are 
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broadly met. 


provider are 












substantially 












met. 
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Annex M: The Oral Feedback Meeting Agenda 

The meeting is expected to last around 30 minutes and is chaired by the review chair. 



1 . Brief account of the range of activities 

2. Announcement of the outcomes of the review, summarising for each 
aspect the main strengths and any weaknesses 

3. Conclusions and any issues that the institution may wish to address 

4. Any points for clarification 

5. Timetable for the production of the subject review report. (The institution 
receives the draft report to comment on the factual accuracy, after the 
review chair has incorporated the specialist reviewers' comments on the 
first draft and following the verification of all subject reviews and draft 
reports, and as soon as possible receives from the HFE the final copy 
together with a summary report containing the graded profile and a copy 
of the subject overview report.) 

6. Thanks offered to the institution for its cooperation and hospitality. 
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Annex N: The Role of the Institutional Facilitator 
Introduction 

Each institution is invited by the HFE to nominate a senior member of staff to take on the role 
of institutional facilitator. The purpose of this role is to liaise closely with the review team to 
ensure that the team obtains accurate and comprehensive information about the educational 
provision and its institutional context. The facilitator will not make judgements on the quality of 
education under review. The experience gained by the facilitator could enable him/her to 
assist the institution to: prepare for review; disseminate good practice; and focus on areas for 
improvement identified by each review. The facilitators will be briefed for their role by the 
HFE. The facilitator can also play an important part after the subject review to facilitate the 
dissemination of review processes within the institution and the action planning that can 
optimise the impact of subject review in the faculty/school after the site-visit. 

During the subject review site-visit, the institutional facilitator's role is to help the peer 
reviewers come to a clear and accurate understanding of the institution's structures, policies, 
priorities and procedures and the nature of provision under review. The facilitator may wish to 
bring additional information to the attention of the reviewers and may seek to correct factual 
inaccuracy. However, it is for the reviewers to decide how best to use this information. 

1. Institutional facilitator activities before visits 

Institutions will find it helpful to involve institutional facilitators fully, and as early as possible, 
in the process of preparing for a subject review visit. Facilitators can make a useful 
contribution to the self-evaluation process as an internal consultant wit ha degree of 
detachment and knowledge of the review process. In particular it is essential that the 
institutional facilitator has access to all correspondence with HFE and the review chair, and 
that the institutional facilitator joins with the preliminary e-mailing discussions. This virtual 
meeting is a valuable first opportunity for review chair and institutional facilitators to discuss 
the latter's role during the visit. It would be helpful to the institutional facilitator if higher 
education institutions could supply the review chair with a brief outline of the institutional 
facilitator's previous experience and current role in the institution, as part of the HFE's 
introductions of key participants in the subject review. 

2. Institutional facilitator during the subject review site-visit 

The primary role of the institutional facilitator is to liaise closely with the review team, to 
ensure that reviewers obtain accurate and comprehensive information about the educational 
provision and its institutional context. The role requires the institutional facilitator to observe 
objectively, communicate clearly, respect the protocols on confidentiality (see below), and 
establish relationships of trust with the review chair, the subject review team, and the named 
departmental or institutional representative. The institutional facilitator is not a member of the 
team, and will not be expected to make judgements on the quality of education under review. 

The institutional facilitator's essential responsibility is to the integrity of the subject review 
process; institutional facilitators should not act as advocates for the subject being reviewed, 
but will provide a means whereby both the institution and the peer reviewers may be confident 
that all appropriate evidence has been made available to subject reviewers. In fulfilling this 
role, the institutional facilitator will: 

• help the institution understand what issues are of concern to subject reviewers 

• respond to requests for comment and information 

• draw the attention of the team to evidence or matters which they may have 
overlooked 

• guide the team on the location of evidence 

• advise the team on institutional policy and context. 

The main vehicle for fulfilling the role described in section 2 will be attendance at meetings. 
The institutional facilitator may attend the following meetings: 
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• All team meetings, except the team meeting held on the final day of the visit to 
determine grades. The institutional facilitator may also attend, if available, any ad hoc 
team meetings. The role of the institutional facilitator at the whole team meeting the team 
meeting will be essentially that of an observer, but contributions to the discussion may be 
made within the context of the institutional facilitator's remit, subject to the discretion of 
the review chair. The review chair will ensure that the role of the institutional facilitator is 
clearly established at the first team meeting. 

• All meetings held between the reviewers and the institution, except meetings with 
current and former students or employers. The role of the institutional facilitator in such 
meetings will be as described above as an observer, except that there may be occasions, 
for example in a meeting to discuss quality management and enhancement, when the 
institutional facilitator's institutional function may need to take precedence over 
facilitation. Any such potential change of role must be clarified with the review chair. 

• All daily meetings between the review chair and the departmental representative. 
These meetings can present a useful opportunity to review the progressive clarification of 
issues. 

The institutional facilitator should be available throughout the duration of the visit. 
In addition to attendance at meetings, the institutional facilitator may require occasional 
access to the base room to convey further information or documentation, or to engage in 
discussions with members of the review team. Such additional access will be at the discretion 
of the review chair, but would normally be readily permitted. Outside these specific cases, the 
institutional facilitator will not normally remain in the base room. 

3. Confidentiality 

The institutional facilitator will observe the same conventions of confidentiality as subject 
reviewers. In particular, no information gained in the course of the review can be used or 
transmitted if it would allow an individual from the institution or the review team to be 
identified. Institutional facilitators will therefore wish to exercise some care when reporting 
back to staff in the faculty/school and will need to remember that subject review is a team- 
based process. The institutional facilitator will have access to the initial written commentaries 
produced by reviewers for the first team meeting, and any initial drafts of report sections that 
are produced for the meeting on the penultimate day, but no documents generated by 
members of the team may be removed from the base room. However, it is perfectly 
acceptable for institutional facilitators to make their own notes on team discussions in order to 
assist the institution's understanding of the issues that are of concern to reviewers. In this and 
other ways, information which relates to the visit and the provision generally can be used or 
transmitted to aid the effectiveness of the visit and to assist institutional facilitators to 
discharge their responsibility for quality enhancement in the institution. 

4. Relations between the institutional facilitator, review chair and 
departmental representative 

The organisation and running of a review visit are the responsibility of the review chair, and 
the responsibility for ensuring that the team is provided with sufficient appropriate evidence to 
allow it to reach its judgements remains with the departmental representative. The role of the 
institutional facilitator in this regard is to ensure that the channels of communication between 
the review chair and subject staff are working effectively. It is not intended that institutional 
facilitators should become responsible for meeting all the reviewers' practical needs. Frequent 
discussions between institutional facilitators and review chair will be normal features of most 
visits, particularly to ensure that institutions have a clear picture of the issues that concern the 
team, and the nature of the evidence required to clarify them. 
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5. The institutional facilitator after the subject review 

The facilitator may wish to play a positive part in the post-review phase when the 
faculty/school receives the review report and considers any action planning that might be 
desirable. This will be of particular value if the outcome of a review signals low grades and the 
possibility of a follow-up review. Equally, the facilitator may have a professional interest in 
high grades denoting good practice that could be transferred to other parts of the institution. 

In addition, institutions may wish the facilitator to assist with promoting awareness within the 
institution of the subject review process and promoting quality enhancement activities derived 
from the experience. These activities are at the discretion of the institution. 

During follow-up review, the facilitator may continue to play a facilitating role in support of the 
follow-up review process guided by the role under subject review as set out above. However, 
if they are thought to be too closely associated with the decisions taken to manage changes 
arising from the subject review to undertake the facilitator role with impartiality, they may 
withdraw from the immediate follow-up review process and the institution in these 
circumstances may nominate another facilitator. 



6. Personal specification for the institutional facilitator 

Essential: 

• thorough knowledge of the structure, policies, priorities, procedures and practices of 
their own institution 

• extensive knowledge and experience of working in higher education at a senior level 

• extensive experience of quality assurance procedures in higher education 

• knowledge and understanding of the process of subject review or the equivalent 

• educated to degree level or equivalent 

• qualifications and experience in a subject area other than that being reviewed 

• able to maintain confidentiality. 

Desirable: 

• direct experience of teaching within higher education 

• experience of managing and leading quality enhancement activities. 
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